What is the Scandium Dataset?

A harmonized dataset of 267,230 materials from Materials Project, OQMD, and JARVIS with quality scoring and provenance tracking.

Can I use the Scandium Dataset commercially?

It depends on the entries. MP entries (CC BY 4.0) and JARVIS entries (CC0) are commercial-safe. OQMD entries are non-commercial. A Commercial-Safe edition script extracts only MP + JARVIS entries.

What is in the SSB Electrolyte Dataset?

About 30,838 bulk structural and thermodynamic DFT entries for solid-state battery electrolytes, including 183 human-verified conductivity and activation-energy labels.

Scandium Datasets — 267,230 Materials (CC BY 4.0)

Name: Scandium Dataset v1.0
Creator: Scandium Labs
License: https://creativecommons.org/licenses/by/4.0/

WHY THIS DATASET

AI Needs Better Data,
Not Just Better Models

The biggest bottleneck in AI-driven materials discovery is no longer architecture — it's data. Most materials datasets are source-specific, lack quality scoring, and don't track provenance. Scandium Dataset solves this by harmonizing three major DFT databases into a single, quality-scored, provenance-tracked resource.

⚛

Cross-Source Harmony

Materials from MP, OQMD, and JARVIS are standardized to shared schema, units, and property definitions. No more manual mapping between database APIs.

★

Quality You Can Trust

Every entry is scored across 5 sub-scores. Calibrated against cross-source agreement. Score 80+ correlates with research-grade label quality.

↟

Provenance Built In

Every entry tracks its source, source_id, checksum, and repair history. Know exactly where each value came from and how it was processed.

DATASET STATISTICS

By the Numbers

A comprehensive view of the dataset composition, quality distribution, and coverage across sources.

267,230

Total Entries

148,141

Unique Formulas

10

Material Families

8.42 GB

Total Dataset Size

TIER DISTRIBUTION

Strict Gold: 56,966

Gold: 96,242

Validated: 140,382

Raw: 30,108

Experimental: 498

PROPERTY COVERAGE

Property	Coverage	Notes
Formation energy (eV/atom)	267,230 (100%)	From MP, OQMD, JARVIS-DFT
Energy above hull (eV/atom)	257,799 (96.5%)	14,669 JARVIS entries computed via internal convex hull
Band gap (eV)	267,079 (99.9%)	151 OQMD entries with non-converged band gap
Space group	267,214 (100%)	spglib symmetry analysis
Volume (Å³)	267,230 (100%)	From structure
Density (g/cm³)	267,230 (100%)	From elements + volume
Structure JSON	267,230 (100%)	pymatgen Structure serialization
Quality score	267,230 (100%)	Composite score (0–88)
Provenance	267,230 (100%)	Source, source_id, checksum
SSE family	267,230 (100%)	Composition-based heuristic

DATA PIPELINE

From Raw APIs to
Ready-to-Use Dataset

Every entry in the Scandium Dataset passes through a rigorous pipeline: ingestion, repair, deduplication, quality scoring, and tier classification.

I.

Source Ingestion

Data is pulled from Materials Project, OQMD, and JARVIS-DFT APIs with checksum verification. Raw archives are frozen and never modified.

II.

Coordinate Repair

OQMD coordinate artifacts detected and repaired. 137,405 entries with known coordinate issues undergo Type A repair with 100% success rate.

III.

Deduplication

Two-pass deduplication (intra-source + cross-source). 31,997 duplicate entries removed across 21,140 duplicate groups. Cross-source pairs identified.

IV.

Quality Scoring

Hierarchical scoring with 5 sub-scores: geometry, DFT convergence, metadata completeness, novelty, and chemical plausibility. Calibrated against cross-source agreement.

V.

Tier Classification

11-gate Strict Gold system. Entries classified as Strict Gold, Gold, Validated, Raw, or Experimental Gold based on validation gates passed.

VI.

Export & Release

Typed Parquet export with indexed lookup. Partitioned by source for selective loading. SHA256-checksummed releases with full MANIFEST.

DATA SOURCES

Three Foundations,
One Dataset

Each source contributes unique strengths. Together they provide comprehensive coverage of the inorganic crystal landscape.

Materials Project

69,279

CC BY 4.0

Highest data quality, complete metadata, reliable provenance. Serves as the gold standard reference.

OQMD

171,780

Non-Commercial

Largest source by volume. Required coordinate repair (137k entries) and symmetry computation (171k space groups).

JARVIS-DFT

25,673

CC0

Commercially safe (CC0). Missing EaH for all entries — computed via internal convex hull in v1.0.0.

OBELiX

498

Per-Article

Experimental Li-ion conductivity data. Integrated as experimental_gold tier with measured transport properties.

AVAILABLE PROPERTIES

Every Property
You Need

47 columns of structural, thermodynamic, electronic, and provenance data. Standardized units and null semantics documented in SCHEMA.md.

Thermodynamic

Formation energy (eV/atom)
Energy above hull (eV/atom)
Band gap (eV)
Magnetic ordering

Structural

Space group (spglib)
Crystal system
Volume (Å³)
Density (g/cm³)
Nsites
Structure JSON

Metadata & Provenance

Source (MP/OQMD/JARVIS)
Source ID
Checksum
Quality score (0–88)
Quality tier
SSE family classification

QUICKSTART

Load the Dataset
in One Line

The dataset is distributed as typed Parquet files for fast, memory-efficient loading.

Inspect schema — no download needed Python

import pyarrow.parquet as pq
schema = pq.read_schema("dataset/entries_v4_typed_sample.parquet")
for f in schema:
    print(f"{f.name}: {f.type}")

Load full dataset with Pandas Python

import pandas as pd
df = pd.read_parquet("dataset/entries_v4_typed.parquet")
print(df.shape, list(df.columns))

Stream in batches — low memory mode Python

pf = pq.ParquetFile("dataset/entries_v4_typed.parquet")
for batch in pf.iter_batches(batch_size=1000, columns=["formula","band_gap"]):
    df_chunk = batch.to_pandas()
    print(f"Loaded {len(df_chunk)} rows")

Filter by source before loading Python

df_mp = pd.read_parquet("dataset/entries_v4_typed.parquet",
    filters=[("source","=","mp")])

DOWNLOAD

Get the Data

Choose the format that works best for your workflow. All files are checksum-verified.

Full Dataset (Parquet)

47 columns, typed, indexed

178 MB · entries_v4_typed.parquet

Download

Sample (1,000 rows)

Stratified sample for schema inspection

1.25 MB · entries_v4_typed_sample.parquet

Download

Battery Subset

82,925 battery-relevant entries

830 MB · battery_subset_v1.json

Download

Electrolyte Subset

41,665 strict Gold electrolyte candidates

492 MB · solid_electrolyte_candidate_subset_v1.json

Download

View on Hugging Face Dataset Hub

LICENSE & TERMS

Not Uniformly Licensed

Each entry carries its own license based on its source. Understand the terms before using the dataset commercially.

MP Entries

69,279

CC BY 4.0

Commercial safe. 26.1% of dataset.

JARVIS Entries

25,673

CC0 1.0

Commercial safe. 9.6% of dataset.

OQMD Entries

171,780

Non-Commercial

Non-commercial only. 64.3% of dataset.

Commercial-Safe Edition

94,952

MP + JARVIS

Extract via included script.

⚠ See LICENSE_BREAKDOWN.md for full details. A Commercial-Safe edition (MP + JARVIS, ~94,952 entries) is extractable via scripts/extract_commercial_safe_edition.py.

CITATION

How to Cite

If you use the Scandium Dataset in your research, please cite it.

BibTeX citation

@misc{scandium-dataset-v1,
    title  = {Scandium Dataset v1.0: A Harmonized Quality-Scored Foundation of DFT-Computed Properties for AI-Driven Materials Discovery},
    author = {Shamique Khan and Scandium Labs},
    year   = {2025},
    howpublished = {\url{https://huggingface.co/datasets/Scandium-Labs/Scandium-Dataset}},
    note   = {CC BY 4.0 (MP entries), CC0 1.0 (JARVIS entries), Non-commercial (OQMD entries)}
}

DOI pending — Zenodo archival in progress.

FAQ

Common Questions

What can this dataset be used for? ×

Phase stability screening, structural family classification, band gap prediction, materials-informatics benchmarking, pretraining property predictors on inorganic crystal structures, and cross-source DFT property harmonization studies.

What should I NOT use this dataset for? +

Direct ionic conductivity prediction, SSE performance ranking, or electrochemical stability assessment. The dataset contains migration barrier proxies for only 23% of Li/Na entries. It provides thermodynamic screening data, not transport property labels.

Can I use this dataset commercially? +

It depends on the entries you use. MP entries (CC BY 4.0) and JARVIS entries (CC0) are commercial-safe. OQMD entries (non-commercial + attribution) are not. A Commercial-Safe edition script is included to extract only MP + JARVIS entries.

How was quality scoring calibrated? +

Quality scores were calibrated against cross-source agreement. For example, MP-JARVIS overlap (11,741 formulas) shows FE MAE 0.20 eV and BG MAE 0.29 eV across sources. Score 80+ entries demonstrate monotonic improvements across 6 validation metrics — establishing research-grade reliability thresholds.

What is the difference between Gold and Strict Gold? +

Gold (96,242 entries) passes 8 validation gates including valid structure, unique composition, stable targets, and complete metadata. Strict Gold (56,966) adds 3 additional gates: quality score ≥ 80 (calibrated), complete provenance with checksum, and no known defects (volume > 0, density > 0, space group present).

Why are OQMD space groups now available? +

OQMD originally did not store symmetry data. A parallel spglib pass was run against all 171,780 OQMD structures, finding space groups for 171,764 entries (16 failed). This unblocked Gold tier classification for 29,068 OQMD entries that were previously stuck in Raw tier.

How do I load only MP data? +

Use Parquet filters: pd.read_parquet("dataset/entries_v4_typed.parquet", filters=[("source","=","mp")]). The dataset is also partitioned by source in the partitioned/ directory.

VERSION HISTORY

Changelog

Track the evolution of the Scandium Dataset across releases.

v1.0.0

Current Release

Harmonized MP + OQMD + JARVIS + OBELiX. 267,230 entries. BVSE migration barriers for 23% of Li/Na. CC BY 4.0 / CC0 / Non-commercial licensing. Parquet format with typed columns. Frozen benchmark splits.

v3.0-rc

Release Candidate

OQMD fixes applied (171,764 space groups, 47,807 volume/density). Strict Gold definition finalized. Quality score calibration completed. Cross-source agreement study published.

v2.0

Quality Scoring Introduced

Hierarchical quality scoring with 5 sub-scores. Tier classification (Gold/Validated/Raw). Cross-source deduplication pipeline established.

v1.0

Initial Release

First public release. Combined MP + OQMD + JARVIS. Basic validation and deduplication. JSON format.

WHY THIS DATASET

The Missing Labels for
SSE Discovery

The critical bottleneck in ML-driven solid electrolyte discovery is the extreme scarcity of reliable experimental room-temperature ionic conductivity labels. Computational structures are abundant, but physical conductivity measurements require complex pellet synthesis, sintering, and electrochemical impedance spectroscopy (EIS). Scandium Labs SSB Dataset addresses this with sentence-level literature evidence, deterministic red-flag verification, and cross-paper consensus.

§

Sentence-Level Evidence

Experimental σ_RT and E_a values are mined from peer-reviewed publications and hand-verified against verbatim sentences, pages, and table or figure annotations with DOIs.

⚑

Red-Flag Verification

Every literature record passes Arrhenius consistency screens (σ₀ ∈ [10¹, 10⁵] S/cm), digit-matching against source text, unit standardization, and copy-paste anomaly detection.

✳

Cross-Paper Consensus

Material-level aggregation tracks independent measurement agreement (n ≥ 3 papers) with statistical outlier bounds, surfacing only high-confidence conductors.

DATASET STATISTICS

By the Numbers

A comprehensive view of the dataset composition, verified transport labels, and coverage across source databases.

30,838

DFT Records

183

Verified Labels

9

Consensus Materials

98%

Silver Tier

HUGGING FACE CONFIGS

default

30.8k rows · full structural + thermodynamic backbone

verified

183 rows · human-reviewed experimental transport labels

consensus

427 rows · materials with independent measurement agreement

gold_benchmark

165 rows · curated benchmark set for supervised learning

DATA SOURCES

Source	Role	Notes
Materials Project	DFT backbone	21,528 rows · CC BY 4.0
JARVIS	DFT backbone	Li-containing structures
NOMAD	DFT backbone	Open archive entries
AFLOW	DFT backbone	150 rows restricted to non-commercial use
OQMD	DFT backbone	50 rows · CC BY 4.0
COD / Materials Cloud	DFT backbone	Structure archive entries
Literature mining	Transport labels	OpenAlex / Unpaywall discovery, human-reviewed

MATERIAL FAMILIES

All 8 SSB Electrolyte
Families Covered

Each family is tagged with expected room-temperature ionic conductivity and activation-energy ranges plus its key transport feature.

Family	Examples	σ_RT (S/cm)	E_a (eV)	Key Feature
Garnet	Li₇La₃Zr₂O₁₂ (LLZO)	10⁻⁵ – 2×10⁻³	0.20 – 0.55	Li-metal stability
NASICON	Li_1+xAl_xTi_2−x(PO₄)₃ (LATP)	10⁻⁵ – 10⁻²	0.20 – 0.45	High air stability
Sulfide	Li₁₀GeP₂S₁₂ (LGPS)	10⁻⁵ – 10⁻¹	0.10 – 0.50	Very high RT conductivity
Argyrodite	Li₆PS₅Cl	10⁻⁴ – 10⁻¹	0.15 – 0.50	Ductile, low GB resistance
Perovskite	Li_3xLa_2/3−xTiO₃ (LLTO)	10⁻⁶ – 10⁻³	0.25 – 0.50	High bulk conductivity
Halide	Li₃YCl₆, Li₃InCl₆	10⁻⁴ – 10⁻²	0.25 – 0.50	High oxidative stability
Oxide	Lithium metal oxides	10⁻¹⁰ – 10⁻²	0.20 – 0.90	Structural foundation
Hydride	LiBH₄, Li₂NH	10⁻⁸ – 10⁻³	0.30 – 0.80	Superionic transitions
Borohydride	LiCB₁₁H₁₂	10⁻⁸ – 10⁻³	0.20 – 1.70	Rotational anion disorder
Antiperovskite	Li₃OCl, Li₃OBr	10⁻⁸ – 10⁻⁴	0.30 – 0.70	Low melting point
Polymer Composite	PEO-LiTFSI + LLZO/LATP	10⁻⁸ – 10⁻³	0.30 – 1.50	Flexible mechanical interface

DATA PIPELINE

From Papers to
Verified Labels

A multi-tier pipeline that unifies literature-mined experimental transport values with bulk DFT structural backbones under deterministic quality control.

I.

Literature Discovery

OpenAlex and Unpaywall discovery of solid-state electrolyte publications. Gold papers catalogued with DOIs, journals, and full-text acquisition.

II.

Literature Extraction

Experimental σ_RT and E_a mined with vision models and human review, tracking evidence to verbatim sentences, pages, tables, and figures.

III.

Red-Flag Verification

Deterministic screens: Arrhenius consistency (σ₀ ∈ [10¹, 10⁵] S/cm), digit-matching against source text, unit standardization, and copy-paste anomaly detection.

IV.

Bulk DFT Backbone

Over 30,000 Li-containing structures pulled across 8 database connectors (Materials Project, JARVIS, NOMAD, AFLOW, OQMD, COD, Materials Cloud) with thermodynamic and structural properties.

V.

Cross-Paper Consensus

Material-level aggregation tracking independent measurement agreement (n ≥ 3 papers) with statistical outlier bounds and agreement grading.

VI.

Quality Gates & Release

10 automated release gates (22/22 passing), 869 passing tests, and quality tiering (98.0% silver). Auto-synced Hugging Face publication at v1.9.0.

AVAILABLE PROPERTIES

Every Property
You Need

Named-schema columns across structural, thermodynamic, ion transport, synthesis, experiment, ML, and provenance blocks.

Ion Transport

Sigma at RT (σ_RT, S/cm)
Activation energy (E_a, eV)
σ vs T curve
Conductivity type & measurement method
Temperature range measured
Mobile ion

Structure & Thermodynamics

Space group & crystal system
Li-site occupancy & coordination
Formation energy (eV/atom)
Energy above hull
Band gap (eV)
Density, volume, nsites

Provenance & ML

Source DOI & journal
Extraction method & confidence score
Evidence sentence / page / table
Family & confidence tier
Split assignment & split group key
Discovery labels (is_good_ssb, etc.)

QUICKSTART

Load the Dataset
in One Line

Load verified experimental transport records or query the consensus database directly from the GitHub repository.

Load human-verified experimental transport records Python

import pandas as pd

df = pd.read_parquet("quality_output/quality.parquet")
verified = df[df["human_verified"] == True]
print(f"Loaded {len(verified)} human-verified experimental transport records.")
print(verified[["composition", "family", "property", "value", "unit", "doi"]].head(10))

Query cross-paper consensus (n ≥ 3 papers) Python

import json

with open("literature_output/consensus_db.json") as f:
    consensus_db = json.load(f)

for comp, data in consensus_db.items():
    if data.get("n_papers", 0) >= 3:
        print(f"Material: {comp:<25} | Papers: {data['n_papers']} | Agreement: {data.get('agreement_grade')}")

Stream from Hugging Face Python

from datasets import load_dataset

ds = load_dataset("Scandium-Labs/solid-state-electrolyte-conductivity", "verified")
print(ds["train"].to_pandas().head())

DOWNLOAD

Get the Data

Access the dataset on Hugging Face or clone the full repository with the pipeline, documentation, and output artifacts.

Hugging Face Dataset

Multi-config: default / verified / consensus / gold_benchmark

Parquet · Scandium-Labs/solid-state-electrolyte-conductivity

Download

GitHub Repository

Full pipeline, docs, quality reports, and output artifacts

ScandiumLabs-in/Scandium-Labs-Solid-State-Battery-Dataset

Clone

Verified Experimental Labels

183 human-reviewed σ_RT / E_a records with sentence-level provenance

quality_output/quality.parquet · human_verified = True

Download

View on Hugging Face Dataset Hub View on GitHub

LICENSE & TERMS

CC BY 4.0 with
Source Carve-Outs

Scandium-authored content is CC BY 4.0 (share and adapt, even commercially, with attribution). Structural data sourced from third-party databases retain their respective source licenses — check identity.source_db per row before assuming redistribution rights.

Materials Project

21,528 rows · CC BY 4.0 · commercial safe

OQMD

50 rows · CC BY 4.0 · commercial safe

AFLOW

150 rows · academic / non-commercial only

Scandium Labels

CC BY 4.0 · literature-derived transport labels

⚠ See LICENSE_BREAKDOWN.md for the full per-source terms.

CITATION

How to Cite

If you use the SSB Electrolyte Dataset in your research, please cite it.

BibTeX citation

@dataset{scandium_ssb_dataset_2026,
    author       = {Scandium Labs Team},
    title        = {Scandium Labs Solid-State Battery (SSB) Transport Dataset},
    year         = {2026},
    version      = {v1.9.0},
    publisher    = {GitHub / Hugging Face},
    url          = {https://huggingface.co/datasets/Scandium-Labs/solid-state-electrolyte-conductivity}
}

FAQ

Common Questions

What is the difference between the Hugging Face configs? ×

default (30.8k rows) is the full structural and thermodynamic DFT backbone. verified (183 rows) contains human-reviewed experimental σ_RT and E_a labels. consensus (427 rows) tracks materials with independent measurement agreement. gold_benchmark (165 rows) is a curated supervised-learning benchmark.

How were the transport labels verified? +

Every experimental value is mined from peer-reviewed publications and hand-verified against verbatim sentences, pages, and table or figure annotations, with DOIs recorded. Each record passes deterministic Arrhenius consistency screens, digit-matching, unit standardization, and copy-paste anomaly detection.

What fraction of entries actually carry transport labels? +

Honest caveat: of the ~30,838 records, only 183 carry human-verified conductivity / activation-energy labels. The remainder are structural and thermodynamic DFT records without transport labels. Quality-tier distribution: silver 98.0%, rejected 2.0%.

Can I use this dataset commercially? +

Scandium-authored content (including the literature-derived transport labels) is CC BY 4.0, which permits commercial use with attribution. Structural rows sourced from AFLOW (150 rows) are restricted to scientific / academic / non-commercial use. Consult identity.source_db per row before assuming redistribution rights.

Which databases feed the DFT backbone? +

Materials Project, JARVIS, NOMAD, AFLOW, OQMD, COD, and Materials Cloud — 8 database connectors providing over 30,000 Li-containing structures with thermodynamic and structural properties.

VERSION HISTORY

Changelog

Track the evolution of the SSB Electrolyte Dataset across releases.

v1.9.0

Current Release

Multi-config Hugging Face publication. 30,838 DFT records + 183 human-verified transport labels. 22/22 release gates passing, 869 tests passing. Auto-synced from the release pipeline.

v0.4.0

Consensus & Benchmarks

Cross-paper consensus engine, gold benchmark split, and quality tiering introduced.

v0.1.0

Initial Release

First public release of literature-derived transport labels with sentence-level evidence tracking.

MATERIALS DATA

Scandium Dataset v1.0

AI Needs Better Data,Not Just Better Models

Cross-Source Harmony

Quality You Can Trust

Provenance Built In

By the Numbers

From Raw APIs toReady-to-Use Dataset

Source Ingestion

Coordinate Repair

Deduplication

Quality Scoring

Tier Classification

Export & Release

Three Foundations,One Dataset

Every PropertyYou Need

Load the Datasetin One Line

Get the Data

Full Dataset (Parquet)

Sample (1,000 rows)

Battery Subset

Electrolyte Subset

Not Uniformly Licensed

How to Cite

Common Questions

Changelog

Current Release

Release Candidate

Quality Scoring Introduced

Initial Release

SSB Electrolyte Dataset

The Missing Labels forSSE Discovery

Sentence-Level Evidence

Red-Flag Verification

Cross-Paper Consensus

By the Numbers

All 8 SSB ElectrolyteFamilies Covered

From Papers toVerified Labels

Literature Discovery

Literature Extraction

Red-Flag Verification

Bulk DFT Backbone

Cross-Paper Consensus

Quality Gates & Release

Every PropertyYou Need

Load the Datasetin One Line

Get the Data

Hugging Face Dataset

GitHub Repository

Verified Experimental Labels

CC BY 4.0 withSource Carve-Outs

How to Cite

Common Questions

Changelog

Current Release

Consensus & Benchmarks

Initial Release

AI Needs Better Data,
Not Just Better Models

From Raw APIs to
Ready-to-Use Dataset

Three Foundations,
One Dataset

Every Property
You Need

Load the Dataset
in One Line

The Missing Labels for
SSE Discovery

All 8 SSB Electrolyte
Families Covered

From Papers to
Verified Labels

Every Property
You Need

Load the Dataset
in One Line

CC BY 4.0 with
Source Carve-Outs