feedforge

feedforge makes XML or CSV product feeds for Heureka, Zbozi.cz and Google Merchant from a JSON catalog snapshot. The program runs offline. The output is deterministic. Products stream through memory one at a time, so large catalogs need little memory.

Optional modes stay opt-in: delta feeds write only added and changed products, and the image checker tests image URLs with HTTP requests.

The package contains a library and the feedforge command.

Install

cargo install feedforge

Input catalog

The input is one JSON object. The object has a shop header and a products list. The shop key must come before the products key, because the reader streams products while it parses.

{
  "shop": {
    "name": "Example Shop",
    "url": "https://example.test",
    "email": "shop@example.test"
  },
  "products": [
    {
      "id": "SKU-1",
      "title": "Kávovar",
      "description": "Malý kávovar pro dvě osoby.",
      "price_minor": 12345,
      "currency": "CZK",
      "availability": "in_stock",
      "delivery_days": 2,
      "url": "https://example.test/p/sku-1",
      "image_urls": ["https://example.test/img/sku-1.jpg"],
      "category_path": ["Domov", "Kuchyně"],
      "ean": "8591234567890",
      "brand": "Acme",
      "weight_grams": 1500,
      "params": {"Barva": "červená"}
    }
  ]
}

Field	Type	Description
id	string	Product identifier, unique in the shop.
title	string	Product title.
description	string	Full product description.
price_minor	integer	Price in minor units, for example haléře or cents.
currency	string	Price currency.
availability	string	in_stock, out_of_stock, preorder or discontinued.
delivery_days	integer	Delivery time in days. Optional. Heureka and Zbozi write it as DELIVERY_DATE.
url	string	Product page URL. It must start with http:// or https://.
image_urls	list of strings	Product image URLs in display order.
category_path	list of strings	Category path from the root category.
ean	string	European Article Number. Optional.
brand	string	Manufacturer or brand name. Optional.
weight_grams	integer	Shipping weight in grams. Optional. The writers do not emit it.
params	object	Free-form name-value pairs. Optional.

Configuration

The --config option points to a TOML file.

portal = "heureka"
output = "feed.xml"
currency = "CZK"
max_description_chars = 2000
required_fields = ["id", "title", "description", "price", "currency", "availability", "url", "image"]

[availability]
in_stock = "0"

[category_map]
"Domov > Kuchyně" = "Kuchyně a jídlo"

Key	Type	Default	Description
portal	string	required	heureka, zbozi or google.
output	path	none	Output feed path. The --output option replaces it.
currency	string	required	Three-letter code. Heureka and Zbozi accept only CZK.
max_description_chars	integer	2000	Maximum description length in Unicode characters.
availability	table	portal defaults	Maps each internal value to a portal value.
category_map	table	empty	Rewrites category path prefixes.
required_fields	list	portal defaults	Replaces the default required field list.
output_format	string	xml	xml or csv.
delimiter	string	,	One character between CSV fields.
csv_fields	list	all fields	Ordered CSV columns. See CSV output.

Availability mapping

The table below shows the default value for each internal status.

Internal	Heureka	Zbozi	Google
in_stock	0	0	in_stock
out_of_stock	1	-1	out_of_stock
preorder	2	8	preorder
discontinued	3	-1	out_of_stock

You can replace each default value in the [availability] table.

Category mapping

A category_map entry rewrites a category path prefix. Write both sides as path segments with > between them. The longest matching prefix wins. The writer then joins the result with the portal separator.

[category_map]
"Domov" = "Domácí potřeby"
"Domov > Kuchyně" = "Kuchyně a jídlo"

A product with the path Domov > Kuchyně > Hrnce becomes Kuchyně a jídlo | Hrnce for Heureka and Kuchyně a jídlo > Hrnce for Google.

Required fields

The default required list is id, title, description, price, currency, availability, url and image. The setting required_fields replaces this list. A product without images is always skipped, also when the list does not contain image. Every listed value must be one of: id, title, description, price, currency, availability, url, image, ean, brand and category.

Portal output

Data	Heureka and Zbozi	Google
Identifier	ITEM_ID	g:id
Title	PRODUCT, PRODUCTNAME	g:title
Description	DESCRIPTION	g:description
URL	URL	g:link
Images	first image IMGURL, extra images IMGURL_ALTERNATIVE	g:image_link, then g:additional_image_link
Price	PRICE_VAT	g:price
Currency	CURRENCY	inside g:price
Availability	AVAILABILITY, or DELIVERY_DATE when delivery_days is present	g:availability
Category	CATEGORYTEXT	g:product_type
EAN	EAN	g:gtin
Brand	MANUFACTURER	g:brand
Parameters	PARAM entries	not emitted
Condition	not emitted	g:condition = new

A product with delivery_days writes that value in DELIVERY_DATE for Heureka and Zbozi. For Heureka the element replaces AVAILABILITY; for Zbozi the per-offer value replaces the availability mapping. Without delivery_days, the writer keeps the availability-derived output. Google ignores delivery_days.

The Zbozi writer has three differences:

1. The SHOP root carries xmlns="http://www.zbozi.cz/ns/offer/1.0".
2. Zbozi writes the availability mapping in DELIVERY_DATE. Heureka writes it in AVAILABILITY. Both portals write per-offer delivery_days in DELIVERY_DATE.
3. Zbozi writes VAL inside PARAM. Heureka writes PARAM_VAL.

The Google writer makes one RSS 2.0 document with the g namespace.

Prices use integer arithmetic only. 12345 minor units become 123.45. Descriptions truncate at the character limit on a character boundary.

CSV output

Set output_format = "csv" to write a CSV table instead of XML. CSV output uses the same validation, availability mapping and category mapping as XML output.

portal = "heureka"
output = "feed.csv"
currency = "CZK"
output_format = "csv"
delimiter = ";"
csv_fields = ["id", "title", "description", "price", "currency", "availability", "url", "image", "ean", "brand", "category"]

The first row is a header with the column names. Columns follow the csv_fields order. Without csv_fields, all supported fields are written in this order: id, title, description, price, currency, availability, url, image, ean, brand, category.

Values follow RFC 4180: a field with the delimiter, a quote or a line break is quoted, and quotes inside a quoted field are doubled. Rows end with LF. The image column joins all image URLs with ;. Missing optional values become empty fields. Products keep the input order, and skipped products do not appear.

Command line

feedforge --config feed.toml --input catalog.json [--output feed.xml] [--report report.json] [--previous catalog.json] [--check-images] [--fail-on-broken] [--strict]

Option	Description
--config FILE	TOML configuration file. Required.
--input FILE	JSON catalog snapshot. Required.
--output FILE	Output path. Replaces the config output value.
--report FILE	Write a JSON report of the run.
--previous FILE	Previous catalog snapshot. Writes only added and changed products.
--check-images	Check item image URLs with HTTP requests.
--fail-on-broken	Exit with code 1 when any checked image is broken. Requires --check-images.
--strict	Exit with code 1 when any product is skipped.

Exit code	Meaning
0	Success.
1	Success, but a product was skipped in --strict mode or an image was broken in --fail-on-broken mode.
2	Configuration error, input error or I/O error.

Delta feeds

Pass --previous <catalog.json> to write only added and changed products. feedforge compares each product by a stable hash of its mapped output fields. Without --previous, every valid product is written.

• A product is added when its id is absent from the previous snapshot.
• A product is changed when its mapped output fields differ.
• A product is removed when its id was in the previous snapshot but is not in the current catalog. Removed ids appear in the report, sorted.
• A product is unchanged when its mapped output fields match. Unchanged products are not written.

The report gains a delta object with added, changed, removed and unchanged counts. The hash covers all mapped fields, also fields outside the CSV column list. Products without an id cannot be compared and are written as added. Products that fail validation are not part of the comparison. The previous snapshot is a normal catalog JSON file.

{
  "written": 3,
  "skipped": [],
  "delta": {
    "added": 2,
    "changed": 1,
    "removed": ["SKU-9"],
    "unchanged": 7
  }
}

Image checker

--check-images checks the image URLs of written products with HTTP requests after the feed is generated. This mode is opt-in. Without it, feedforge stays offline and makes no network requests.

• Each unique URL is checked once.
• The checker sends a HEAD request. A 405 response falls back to GET.
• Requests run with a concurrency of 4 and a timeout of 10 seconds per request.
• A status of 400 or higher, a timeout or a connection error marks the URL as broken. Redirects are followed.
• Broken URLs appear in the report with the HTTP status or the error text.

--fail-on-broken exits with code 1 when any checked image is broken.

{
  "written": 2,
  "skipped": [],
  "broken_images": [
    {"url": "https://example.test/img/gone.jpg", "status": 404},
    {"url": "https://example.test/img/offline.jpg", "error": "Connection refused (os error 111)"}
  ]
}

Report

The report lists the number of written products and every skipped product with the skip reasons.

{
  "written": 12,
  "skipped": [
    {
      "index": 3,
      "id": "SKU-4",
      "reasons": ["missing_field:description", "invalid_url:image_urls"]
    }
  ]
}

Reason	Meaning
missing_field:NAME	A required field is absent or empty.
invalid_url:url	The product URL is invalid.
invalid_url:image_urls	At least one image URL is invalid.
invalid_price	The price is negative.
currency_mismatch	The product currency differs from the config currency.

A product with an empty image_urls list is always skipped with missing_field:image.

A delta run adds the delta object. An image check run adds the broken_images list. Without these options, the report keeps the shape above.

Determinism

Identical input and configuration produce identical output bytes. Products keep the input order. Parameters sort by name. The output holds no timestamps. The report is deterministic too.

Without --check-images, generation performs no network requests. The image checker depends on remote servers, so its broken_images entries are not deterministic.

Development

cargo fmt --check
cargo clippy --all-targets -- -D warnings
cargo test
cargo build --release

License

MIT. See LICENSE.