Methods

The schemas are the framework

Everything else in this atlas is an instance of one of four things. These files are the contract.

They are JSON Schema 2020-12. Validate with python3 pipelines/validate_entries.py, which also rebuilds ../catalog.json.

The four rules the schemas enforce

1. A number without a source does not exist. Every entry requires at least one source, and a source requires a locator — the page, table, tab or docket item that lets a reader land on the number rather than on the document.

2. Nothing is reaggregated silently. Every dataset declares the grain at which its source reports. Joining across grains is legal, but it happens through a published crosswalk in ../crosswalks/, each of which states its method and its error characteristic. The seam is shown, not hidden.

3. A decision must say who paid. who_paid is a required field on every decision. Where the instrument does not allocate the cost, the field says so — an unallocated cost is a finding, not a blank. The absence field exists for the case the atlas most wants to record: a decision that consists of something never being written down.

4. A dataset must state a limitation. limitations requires at least one entry. A dataset with no stated limitation has not been examined. "Data may contain errors" does not satisfy it; "resolution is the feeder, so no address-level claim is possible" does.

Reliability tiers

An entry's tier is the weakest tier of anything load-bearing inside it, not the average.

Adding a value to a controlled list

Eras, grains, tiers, instrument kinds and site kinds are closed enums in common.schema.json. Adding one is a deliberate act: change the enum, say why in the commit, and re-validate every entry. This is the point — a vocabulary that grows by typo is not a vocabulary.

Portability

Nothing in these schemas is specific to Boston. The era list in ../eras.json is, and it is held in a separate file for exactly that reason: another city adopting this framework replaces eras.json and keeps the contract.

---

Crosswalks — how the atlas joins across grains

The atlas's spine is the distribution feeder and its social join is the census tract. Feeders do not nest inside tracts, ZIPs or municipalities. They cross all of them. So a join across grains is never done silently: it is done through one of these files, each of which states its method and its error, and each row carries a share so a user can see how much of a thing actually falls where.

Rebuild with python3 pipelines/build_crosswalks.py. Stdlib only — no shapely, no geopandas. Where this project has to compute an overlap itself it samples, and states the resolution. Where the Census has already computed the overlap from block geometry and published it, that file is used instead: it is exact, and it is the authority's own answer.

The two Census relationship files, filtered to Massachusetts, are shipped in census/ so the crosswalks rebuild offline. They are U.S. federal public-domain data.

---

point_to_feeder.csv

What it is. Every data centre building and public or city-owned charging station that could be matched to a distribution circuit, with the circuit, its voltage, its bulk supply substation, and the distance from the point to the circuit's geometry.

Method. Nearest circuit by geometry, using a spatial hash grid with longitude scaled by cos(42.36°) = 0.74 so that a degree of longitude and a degree of latitude are comparable at this latitude.

Error characteristic — read this one carefully. The nearest circuit is not necessarily the circuit that serves the point. Eversource does not publish which customer is on which circuit. What this file says is which circuit runs closest, and how close. A 6 m match is strongly suggestive; a 300 m match is little more than a neighbourhood. The distance_m column exists so that a user can set their own threshold rather than inherit one.

Two further limits: the circuit geometry is simplified for web delivery, so a distance of a few metres is inside the simplification error; and 9 of 425 public chargers, and 14 of 25 data centre buildings, had no circuit within range and are absent from the file rather than assigned to a distant one.

---

feeder_to_tract.csv

What it is. For each circuit, the share of its length that runs through each census tract.

Method. Each circuit is sampled at 25 m intervals along its geometry and every sample is assigned to the tract containing it. share_of_circuit_length is the fraction of that circuit's samples falling in that tract. The feeder layer holds one record per circuit per voltage, so samples are pooled by circuit id first — otherwise the 36 two-voltage circuits would get two rows per tract, each describing only part of the circuit.

Error characteristic. Resolution is 25 m, so a tract carrying less than about 25 m of a circuit may be missed entirely. Length is measured on simplified geometry, which shortens a sinuous run slightly. 92.5% of sampled circuit length falls inside a study-area tract; the rest runs through Brookline and other Norfolk and Essex ground the tract layer does not cover, or along long transmission-voltage sections. 987 of 1,026 circuits appear here; the other 39 lie entirely outside Suffolk and Middlesex.

What it is not. Length share is not load share, customer share, or population share. A circuit may run three kilometres through a tract and serve nobody in it. Using this file to attribute a circuit's hosting capacity to a tract's households would be exactly the silent reaggregation the atlas exists to avoid.

---

tract2010_to_tract2020.csv

What it is. How the 2010 census tracts map onto the 2020 ones, for Massachusetts. Straight from the Census Bureau's own 2020 relationship file.

Why it exists, and what it fixed. DOE's LEAD energy-burden data is published on the 2010 tract vintage. The atlas's tracts are 2020. The published layer joined them by GEOID string equality, which works for the 1,326 Massachusetts tracts whose boundaries did not change and silently drops every one that did. In the study area that was 144 of 591 tracts — nearly a quarter — and not a random quarter: tracts are renumbered when population changes enough to split or merge them, so the dropped tracts were the ones that had changed most. Energy-burden coverage went from 441 tracts to 580 when this crosswalk replaced the string join.

Two shares, and they are different numbers.

• alloc_share — the fraction of the 2010 tract lying inside the 2020 tract. This is the weight you multiply a 2010 count by when carrying it forward.
• compose_share — the fraction of the 2020 tract covered by that 2010 piece. This tells you how completely the 2020 tract is described by what you have.

Using the wrong one silently rescales everything. unchanged flags the 1:1 cases, where both are 1.

How burden is composed. Each contributing 2010 tract's households are allocated forward by alloc_share, and the burden is recomputed from the composed dollars — total energy spend over total income — rather than averaged from the ratio. Averaging a ratio across populations of different sizes is its own error.

Error characteristic. The overlap is exact; the allocation assumes households are spread evenly over a tract's land area, which they are not. Where a 2020 tract is composed from several 2010 pieces the figure is a blend, and tract_energy_burden.method and coverage say so per tract. Three Suffolk special-use tracts — the airport and harbour islands — are covered by under 25% of LEAD ground and their values are suppressed rather than mapped.

---

zip_to_tract.csv

What it is. The overlap between 2020 ZIP Code Tabulation Areas and 2020 census tracts, from the Census Bureau's own relationship file. It replaces a 150 m lattice this project used to sample itself; the Census computes this from block geometry, so it is exact rather than estimated.

share_of_source_area is the fraction of the ZCTA in that tract; share_of_tract_area is the fraction of the tract in that ZCTA.

What it is not — and this matters most for the solar data. Area share is not housing share. A ZIP's solar installations sit on rooftops, not spread evenly over ground. Multiplying a ZIP's installation count by an area share to get a tract-level count is arithmetically easy and substantively wrong, and the atlas does not do it.

An earlier version of the data-architecture proposal said the Census relationship file would supply a housing-unit weight. It does not — the 2020 relationship files publish land and water area only. A housing-weighted crosswalk needs either the HUD USPS ZIP-tract file, which requires registration, or block-level housing counts, which need the Census API and a key. Neither was obtained. That is recorded as gap:no-housing-weighted-crosswalk.

---

municipality_to_tract.csv

What it is. The share of each municipality's area falling in each tract, still by this project's own lattice sampling — a 150 m grid, clamped between 24 and 200 samples per side.

Error characteristic. Area shares with a sampling error of roughly ±1/√n. A tract smaller than about 150 m across can be missed. Boston resolves to 207 tracts, Cambridge to 33; 504 of 591 tracts are reached by some municipality, the rest being Middlesex towns outside the 68-municipality solar layer.

It should be exact and is not. Massachusetts census tracts are drawn inside county subdivisions, so municipality-to-tract ought to be plain containment rather than sampling. Replacing it is a small job and has not been done.

---

The rule these files serve

Every dataset declares the grain at which its source reports, and is never silently reaggregated. Joins across grains happen through a published crosswalk that states its method and its error characteristic.

Charter, §4. The seam between grains is shown, not hidden.