Skip to content

Commit e2cbdeb

Browse files
divyang4481divyang4481
authored
Reproducibility and Documentation Improvements for Open-Data Validation (#12)
* Implement reproducibility and documentation improvements for open-data validation. Added: - `xtheta/data/bell_chsh.py`: `compute_chsh_variants` for auditing sign variants. - `scripts/download_open_data.py`: Idempotent download helper for Hensen dataset. - `scripts/run_all_open_data_validation.py`: Batch runner with graceful skip logic. - `notebooks/06_hensen_data_audit.ipynb`: Comprehensive audit notebook for Hensen data. - New tests for CHSH variants, benchmark schema, and batch runner skip logic. - README update with detailed PowerShell and Bash instructions and scientific warnings. * Implement reproducibility and documentation improvements for open-data validation. - Added `compute_chsh_variants` helper in `xtheta/data/bell_chsh.py`. - Created `scripts/download_open_data.py` for idempotent Hensen data download. - Created `scripts/run_all_open_data_validation.py` for batch processing. - Created `notebooks/06_hensen_data_audit.ipynb` for data auditing. - Added tests for CHSH variants, benchmark schema, and batch runner skip logic. - Updated `README.md` with comprehensive PowerShell/Bash instructions and scientific warnings. - Fixed `tests/test_open_data_batch.py` to ensure paths are resolved correctly in CI environments. * Implement reproducibility and documentation improvements for open-data validation. - Added `compute_chsh_variants` helper in `xtheta/data/bell_chsh.py`. - Created `scripts/download_open_data.py` for idempotent Hensen data download. - Created `scripts/run_all_open_data_validation.py` for batch processing. - Created `notebooks/06_hensen_data_audit.ipynb` for data auditing. - Added tests for CHSH variants, benchmark schema, and batch runner skip logic. - Updated `README.md` with comprehensive PowerShell/Bash instructions and scientific warnings. - Fixed `tests/test_open_data_batch.py` to ensure paths are resolved correctly in CI environments. - Updated `README.md` to use a generic placeholder path for PowerShell instructions. ---------
1 parent 0003fca commit e2cbdeb

8 files changed

Lines changed: 456 additions & 35 deletions

File tree

‎xtheta-lab/README.md‎

Lines changed: 61 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -73,47 +73,85 @@ Outputs will be saved in the `outputs/` directory.
7373
pytest tests/ -v
7474
```
7575

76-
## Open Bell/CHSH Data Validation
76+
## Reproducible Open-Data Validation
7777

78-
The framework includes a validation pipeline for real Bell-test datasets. This pipeline computes the CHSH S-statistic and fits an effective phenomenological X-Theta phase ($\Phi_{eff}$) and anisotropy ($R_{\Theta, eff}$).
78+
The framework includes a formal validation pipeline for real Bell-test datasets. This pipeline computes the CHSH S-statistic, calculates sign variants, and fits an effective phenomenological X-Theta phase ($\Phi_{\rm eff}$) and anisotropy ($R_{\Theta, \rm eff}$).
7979

8080
### Scientific Warning
81-
**Phi_eff is an effective phenomenological parameter only.** Without gravitational path, altitude, curvature, or spacetime-baseline metadata, this is not evidence of spacetime-induced X-Theta holonomy.
81+
**Phi_eff is an effective phenomenological parameter only. Without gravitational path, altitude, curvature, or spacetime-baseline metadata, the open Bell/CHSH datasets are not evidence of spacetime-induced X-Theta holonomy. They validate the computational mapping from observed CHSH statistics to effective X-Theta parameters.**
8282

83-
### Running Validation
83+
### 1. Windows PowerShell Instructions
8484

85-
You can run validation on generic CSV data or supported specific datasets (Weihs, Hensen, BIG Bell Test):
86-
87-
```bash
88-
python scripts/run_open_data_validation.py \
89-
--dataset hensen \
90-
--data path/to/dataset.txt \
91-
--output outputs/open_data/hensen
92-
```
85+
All commands assume you are in the project root:
9386

9487
```powershell
95-
$env:PYTHONPATH = "$env:PYTHONPATH;$PWD\xtheta-lab"
96-
python -m pytest xtheta-lab\tests\
97-
```
88+
cd C:\path\to\X-theta-framework\xtheta-lab
9889
99-
## Open Bell/CHSH Data Validation
90+
python -m venv .venv
91+
.venv\Scripts\activate
10092
101-
The framework includes a validation pipeline for real Bell-test datasets. This pipeline computes the CHSH S-statistic and fits an effective phenomenological X-Theta phase ($\Phi_{eff}$) and anisotropy ($R_{\Theta, eff}$).
93+
pip install -r requirements.txt
94+
pip install -e .
10295
103-
### Scientific Warning
104-
**Phi_eff is an effective phenomenological parameter only.** Without gravitational path, altitude, curvature, or spacetime-baseline metadata, this is not evidence of spacetime-induced X-Theta holonomy.
96+
# Download Hensen/Delft 2015 data
97+
python scripts\download_open_data.py --dataset hensen
98+
99+
# Run validation for Hensen
100+
python scripts\run_open_data_validation.py `
101+
--dataset hensen `
102+
--data data\open_bell\hensen\raw\bell_open_data.txt `
103+
--output outputs\open_data\hensen `
104+
--bootstrap-samples 1000
105+
106+
# Run batch runner for all available datasets
107+
python scripts\run_all_open_data_validation.py
108+
109+
# View comparison results
110+
Import-Csv outputs\open_data\comparison\open_data_comparison.csv | Format-Table
105111
106-
### Running Validation
112+
# Run tests
113+
python -m pytest tests\ -v
114+
```
115+
116+
### 2. Bash/Linux/Mac Instructions
107117

108-
You can run validation on generic CSV data or supported specific datasets (Weihs, Hensen, BIG Bell Test):
118+
All commands assume you are in the project root:
109119

110120
```bash
121+
cd X-theta-framework/xtheta-lab
122+
123+
python -m venv .venv
124+
source .venv/bin/activate
125+
126+
pip install -r requirements.txt
127+
pip install -e .
128+
129+
# Download Hensen/Delft 2015 data
130+
python scripts/download_open_data.py --dataset hensen
131+
132+
# Run validation for Hensen
111133
python scripts/run_open_data_validation.py \
112134
--dataset hensen \
113-
--data ../bell_open_data.txt \
114-
--output outputs/open_data/hensen
135+
--data data/open_bell/hensen/raw/bell_open_data.txt \
136+
--output outputs/open_data/hensen \
137+
--bootstrap-samples 1000
138+
139+
# Run batch runner for all available datasets
140+
python scripts/run_all_open_data_validation.py
141+
142+
# View comparison results
143+
cat outputs/open_data/comparison/open_data_comparison.csv
144+
145+
# Run tests
146+
python -m pytest tests/ -v
115147
```
116148

149+
### Expected Outputs
150+
- `outputs/open_data/hensen/data/hensen_chsh_summary.csv`
151+
- `outputs/open_data/hensen/data/hensen_setting_counts.csv`
152+
- `outputs/open_data/hensen/reports/hensen_validation_report.md`
153+
- `outputs/open_data/comparison/open_data_comparison.csv`
154+
117155
### Interpreting Phi_eff
118156
- $\Phi_{eff} \approx 0$ indicates maximal Bell violation ($S \approx 2\sqrt{2}$) and minimal anisotropy.
119157
- $\Phi_{eff} \approx \pi/4$ indicates a result at the classical Bell limit ($S \approx 2$).

‎xtheta-lab/notebooks/06_hensen_data_audit.ipynb‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "code",
5+
"execution_count": null,
6+
"metadata": {},
7+
"outputs": [],
8+
"source": [
9+
"import os\n",
10+
"import sys\n",
11+
"from pathlib import Path\n",
12+
"\n",
13+
"# Add project root to sys.path\n",
14+
"project_root = Path(\"..\").resolve()\n",
15+
"if str(project_root) not in sys.path:\n",
16+
" sys.path.append(str(project_root))\n",
17+
"\n",
18+
"import pandas as pd\n",
19+
"import numpy as np\n",
20+
"from xtheta.data.adapters.hensen import load_hensen_dataset\n",
21+
"from xtheta.data.bell_chsh import compute_chsh_variants\n",
22+
"from xtheta.data.schema import BellEventSchema"
23+
]
24+
},
25+
{
26+
"cell_type": "markdown",
27+
"metadata": {},
28+
"source": [
29+
"# 06 Hensen (Delft) 2015 Open-Data Audit\n",
30+
"\n",
31+
"This notebook audits the raw Hensen et al. (2015) data and our internal adapter mapping to ensure CHSH S calculation accuracy.\n",
32+
"\n",
33+
"**Scientific Warning:**\n",
34+
"Phi_eff is an effective phenomenological parameter only. Without gravitational path, altitude, curvature, or spacetime-baseline metadata, the open Bell/CHSH datasets are not evidence of spacetime-induced X-Theta holonomy. They validate the computational mapping from observed CHSH statistics to effective X-Theta parameters."
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"metadata": {},
40+
"source": [
41+
"## 1. Manual Raw Audit\n",
42+
"\n",
43+
"We inspect the raw file structure directly."
44+
]
45+
},
46+
{
47+
"cell_type": "code",
48+
"execution_count": null,
49+
"metadata": {},
50+
"outputs": [],
51+
"source": [
52+
"raw_path = Path(\"../data/open_bell/hensen/raw/bell_open_data.txt\")\n",
53+
"\n",
54+
"if not raw_path.exists():\n",
55+
" print(f\"[ERROR] Raw data not found at {raw_path}\")\n",
56+
" print(\"Please run: python ../scripts/download_open_data.py --dataset hensen\")\n",
57+
"else:\n",
58+
" print(f\"Raw file path: {raw_path.resolve()}\")\n",
59+
" raw_lines = raw_path.read_text(encoding='utf-8').splitlines()\n",
60+
" print(f\"\\nFirst 10 raw lines:\")\n",
61+
" for line in raw_lines[:10]:\n",
62+
" print(line)"
63+
]
64+
},
65+
{
66+
"cell_type": "markdown",
67+
"metadata": {},
68+
"source": [
69+
"## 2. Adapter Audit\n",
70+
"\n",
71+
"We use the `load_hensen_dataset` adapter to parse the data and verify mapping."
72+
]
73+
},
74+
{
75+
"cell_type": "code",
76+
"execution_count": null,
77+
"metadata": {},
78+
"outputs": [],
79+
"source": [
80+
"if raw_path.exists():\n",
81+
" # Load all data\n",
82+
" df_iter = load_hensen_dataset(str(raw_path))\n",
83+
" df = pd.concat(list(df_iter), ignore_index=True)\n",
84+
" \n",
85+
" print(f\"Parsed DataFrame Preview (first 5 rows):\")\n",
86+
" display(df.head())\n",
87+
" \n",
88+
" schema = BellEventSchema()\n",
89+
" \n",
90+
" print(\"\\nUnique Alice Settings:\", df[schema.alice_setting].unique())\n",
91+
" print(\"Unique Bob Settings:\", df[schema.bob_setting].unique())\n",
92+
" \n",
93+
" print(\"\\nAlice Outcome Counts:\")\n",
94+
" print(df[schema.alice_outcome].value_counts())\n",
95+
" \n",
96+
" print(\"\\nBob Outcome Counts:\")\n",
97+
" print(df[schema.bob_outcome].value_counts())\n",
98+
" \n",
99+
" print(\"\\nSetting-pair counts:\")\n",
100+
" counts = df.groupby([schema.alice_setting, schema.bob_setting]).size()\n",
101+
" print(counts)"
102+
]
103+
},
104+
{
105+
"cell_type": "markdown",
106+
"metadata": {},
107+
"source": [
108+
"## 3. Correlation and CHSH Audit\n",
109+
"\n",
110+
"Calculate expectations $E(a,b)$ and CHSH variants."
111+
]
112+
},
113+
{
114+
"cell_type": "code",
115+
"execution_count": null,
116+
"metadata": {},
117+
"outputs": [],
118+
"source": [
119+
"if raw_path.exists():\n",
120+
" df['ab'] = df[schema.alice_outcome] * df[schema.bob_outcome]\n",
121+
" \n",
122+
" # Calculate E(a,b)\n",
123+
" expectations = df.groupby([schema.alice_setting, schema.bob_setting])['ab'].mean()\n",
124+
" \n",
125+
" E00 = expectations.get((0, 0), 0.0)\n",
126+
" E01 = expectations.get((0, 1), 0.0)\n",
127+
" E10 = expectations.get((1, 0), 0.0)\n",
128+
" E11 = expectations.get((1, 1), 0.0)\n",
129+
" \n",
130+
" print(f\"E00: {E00:.4f}\")\n",
131+
" print(f\"E01: {E01:.4f}\")\n",
132+
" print(f\"E10: {E10:.4f}\")\n",
133+
" print(f\"E11: {E11:.4f}\")\n",
134+
"\n",
135+
" variants = compute_chsh_variants(E00, E01, E10, E11)\n",
136+
" \n",
137+
" print(\"\\nCHSH Sign Variants:\")\n",
138+
" for k, v in variants.items():\n",
139+
" if k not in ['max_abs', 'max_abs_convention']:\n",
140+
" print(f\" {k}: {v:.4f}\")\n",
141+
" \n",
142+
" print(f\"\\nMax Absolute CHSH: {variants['max_abs']:.4f} (Convention: {variants['max_abs_convention']})\")"
143+
]
144+
}
145+
],
146+
"metadata": {
147+
"kernelspec": {
148+
"display_name": "Python 3",
149+
"language": "python",
150+
"name": "python3"
151+
},
152+
"language_info": {
153+
"codemirror_mode": {
154+
"name": "ipython",
155+
"version": 3
156+
},
157+
"file_extension": ".py",
158+
"mimetype": "text/x-python",
159+
"name": "python",
160+
"nbconvert_exporter": "python",
161+
"pygments_lexer": "ipython3",
162+
"version": "3.10.0"
163+
}
164+
},
165+
"nbformat": 4,
166+
"nbformat_minor": 5
167+
}

‎xtheta-lab/scripts/download_open_data.py‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
#!/usr/bin/env python3
2+
"""
3+
Helper script to download and extract open Bell/CHSH datasets.
4+
"""
5+
from __future__ import annotations
6+
7+
import argparse
8+
import os
9+
import urllib.request
10+
import zipfile
11+
from pathlib import Path
12+
13+
HENSEN_ZIP_URL = "https://data.4tu.nl/file/e8cf2991-3153-48ad-b67d-dfd7d7d97fd3/289c4850-6ed5-45a3-8b19-de8671f873a8"
14+
15+
def download_file(url: str, dest: Path):
16+
print(f"Downloading {url} to {dest}...")
17+
urllib.request.urlretrieve(url, dest)
18+
print("Download complete.")
19+
20+
def extract_zip(zip_path: Path, extract_to: Path):
21+
print(f"Extracting {zip_path} to {extract_to}...")
22+
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
23+
zip_ref.extractall(extract_to)
24+
print("Extraction complete.")
25+
26+
def main():
27+
parser = argparse.ArgumentParser(description="Download open Bell/CHSH datasets.")
28+
parser.add_argument("--dataset", choices=["hensen"], required=True, help="Dataset to download.")
29+
parser.add_argument("--force", action="store_true", help="Force re-download and re-extraction.")
30+
31+
args = parser.parse_args()
32+
33+
if args.dataset == "hensen":
34+
base_dir = Path("data/open_bell/hensen")
35+
raw_dir = base_dir / "raw"
36+
zip_path = base_dir / "data.zip"
37+
target_file = raw_dir / "bell_open_data.txt"
38+
39+
base_dir.mkdir(parents=True, exist_ok=True)
40+
41+
# Download
42+
if not zip_path.exists() or args.force:
43+
download_file(HENSEN_ZIP_URL, zip_path)
44+
else:
45+
print(f"Skip download: {zip_path} already exists. Use --force to re-download.")
46+
47+
# Extract
48+
if not target_file.exists() or args.force:
49+
raw_dir.mkdir(parents=True, exist_ok=True)
50+
extract_zip(zip_path, raw_dir)
51+
else:
52+
print(f"Skip extraction: {target_file} already exists. Use --force to re-extract.")
53+
54+
# Final check
55+
if target_file.exists():
56+
print(f"\nSuccess! Hensen data ready at: {target_file}")
57+
print("\nValidation command:")
58+
print(f"python scripts/run_open_data_validation.py --dataset hensen --data {target_file} --output outputs/open_data/hensen --bootstrap-samples 1000")
59+
else:
60+
print(f"\nError: Expected file not found after extraction: {target_file}")
61+
62+
if __name__ == "__main__":
63+
main()

0 commit comments

Comments
 (0)