Methodology

Data Pipeline Process

Data flows through five stages from official source to rendered chart:

1. Fetch Automated Python scripts download raw data files from government and academic sources (ONS, HMRC, WID, BoE, Resolution Foundation).
2. Validate Column presence, data types, and expected ranges are checked. Malformed or missing files halt the pipeline with a clear error.
3. Process Raw data is cleaned, normalised, and written to standardised CSV files. Beyond standard normalisation (e.g. rebasing to an index or converting to real terms), no modelling or extrapolation is applied; where live data is temporarily unavailable the pipeline substitutes clearly-labelled illustrative values.
4. Serve The FastAPI backend reads processed CSVs and exposes them as paginated JSON endpoints with full metadata and source citations.
5. Render The Vue 3 frontend fetches data from the API and renders interactive, accessible, mobile-responsive charts.

Source Citations

All 12 datasets are sourced from official government and academic publications. Each entry below documents the source name, URL, access date, and update frequency.

Data Quality

Every dataset passes automated quality checks before it reaches the API:

• Column validation — expected columns must be present with correct names. Missing or renamed columns halt the pipeline.
• NaN handling — null and missing values are explicitly detected. They are preserved as null in JSON responses rather than silently dropped or filled.
• Type enforcement — numeric columns are verified as numeric; date columns as valid dates. Type mismatches are logged and rejected.
• Row count checks — datasets with zero rows or unexpectedly low row counts trigger warnings.
• Encoding safety — all files are read as UTF-8. Encoding errors are caught and reported rather than producing garbled output.

Limitations

No dataset is perfect. Users should be aware of the following limitations when interpreting the charts:

• Survey sample sizes — the Wealth and Assets Survey samples approximately 20,000 households. Extreme wealth is systematically underrepresented because the very wealthy are less likely to respond. The WAS also lost accredited official statistics status in June 2025 after its response rate fell from 66% to 41%.
• Time lag — official statistics are published months or years after the period they cover. The most recent data point may be 1-3 years old.
• Wealth measurement difficulties — wealth held in trusts, offshore accounts, or complex structures may not appear in survey or administrative data.
• Self-reported data biases — household surveys rely on self-reporting. Respondents may underestimate or overestimate their assets and liabilities.
• Geographic coverage varies — some datasets cover Great Britain only (excluding Northern Ireland), others cover the full United Kingdom, and some are England and Wales only.
• Definitional differences — "wealth" is defined differently across sources (net financial wealth vs. total wealth including property and pensions).

Update Schedule

The data pipeline runs automatically every week via GitHub Actions. Each run fetches the latest available data from these sources, validates it, and updates the processed CSV files. If a source has not published new data since the last run, the existing file is retained unchanged.

Pipeline run logs are publicly visible in the repository's Actions tab. Failed runs trigger alerts and do not update any data files, ensuring stale-but-correct data is always preferred over fresh-but-broken data.

Verification

Every claim on this site can be independently verified:

• Check the source data — every chart links to the original government or academic publication. Click through and compare the numbers yourself.
• Read the pipeline code — all data processing scripts are open source at github.com/Chris0Jeky/wealthlens-hq/automation/data-pipelines.
• Download the processed data — every dataset is published as static CSV and JSON files alongside the site (linked from the home-page cards and each chart's toolbar). Compare our processed output against the raw source.
• Run it yourself — clone the repository and run the pipeline locally. The entire stack is reproducible with standard Python and Node.js tooling.

Privacy

WealthLens UK does not collect personal data. Specifically:

• No personal data is collected, stored, or processed.
• No cookies are used for tracking or advertising.
• No third-party tracking scripts are loaded unless privacy-respecting, aggregate analytics are explicitly enabled for a deployment.
• Analytics, if enabled, are privacy-respecting and aggregated — no individual user profiles are created.
• All datasets contain only aggregate, publicly available statistics. No individual-level data is used.