Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2690 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0715. Min Act: -0.1700

Data Index: 1272470 (Open Web Text)

Max Activating Token Index: 340

Text #1

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0606. Min Act: -0.1700

Data Index: 6997822 (Open Web Text)

Max Activating Token Index: 265

Text #2

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0239. Min Act: -0.1700

Data Index: 3324723 (Open Web Text)

Max Activating Token Index: 548

Text #3

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0367. Min Act: -0.1700

Data Index: 8125109 (Open Web Text)

Max Activating Token Index: 182

Text #4

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0185. Min Act: -0.1700

Data Index: 2627643 (Open Web Text)

Max Activating Token Index: 236

Text #5

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0123. Min Act: -0.1700

Data Index: 2070079 (Open Web Text)

Max Activating Token Index: 193

Text #6

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0094. Min Act: -0.1700

Data Index: 6454856 (Open Web Text)

Max Activating Token Index: 763

Text #7

Max Range: 4.0715. Min Range: -4.0715

Max Act: 4.0124. Min Act: -0.1700

Data Index: 1474988 (Open Web Text)

Max Activating Token Index: 126

Text #8

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9966. Min Act: -0.1700

Data Index: 6997808 (Open Web Text)

Max Activating Token Index: 571

Text #9

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9652. Min Act: -0.1700

Data Index: 5226015 (Open Web Text)

Max Activating Token Index: 876

Text #10

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9680. Min Act: -0.1700

Data Index: 5963807 (Open Web Text)

Max Activating Token Index: 956

Text #11

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9611. Min Act: -0.1700

Data Index: 2341216 (Open Web Text)

Max Activating Token Index: 205

Text #12

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9578. Min Act: -0.1700

Data Index: 7716759 (Open Web Text)

Max Activating Token Index: 339

Text #13

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9741. Min Act: -0.1700

Data Index: 1303226 (Open Web Text)

Max Activating Token Index: 732

Text #14

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9861. Min Act: -0.1700

Data Index: 645194 (Open Web Text)

Max Activating Token Index: 716

Text #15

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9714. Min Act: -0.1700

Data Index: 6997815 (Open Web Text)

Max Activating Token Index: 249

Text #16

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9606. Min Act: -0.1700

Data Index: 5567269 (Open Web Text)

Max Activating Token Index: 868

Text #17

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9875. Min Act: -0.1700

Data Index: 8191285 (Open Web Text)

Max Activating Token Index: 965

Text #18

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9546. Min Act: -0.1700

Data Index: 6777097 (Open Web Text)

Max Activating Token Index: 354

Text #19

Max Range: 4.0715. Min Range: -4.0715

Max Act: 3.9388. Min Act: -0.1700

Data Index: 6777099 (Open Web Text)

Max Activating Token Index: 542