Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3056 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.6433. Min Act: -0.1608

Data Index: 5462485 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.6396. Min Act: -0.1246

Data Index: 8503618 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.6439. Min Act: -0.1222

Data Index: 4924714 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.6261. Min Act: -0.1201

Data Index: 6777412 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.6052. Min Act: -0.1371

Data Index: 8491717 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5887. Min Act: -0.1399

Data Index: 3885738 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5918. Min Act: -0.1077

Data Index: 6085790 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5794. Min Act: -0.1270

Data Index: 919344 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5864. Min Act: -0.1444

Data Index: 3912265 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5796. Min Act: -0.1234

Data Index: 1090731 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5775. Min Act: -0.1442

Data Index: 2029593 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5736. Min Act: -0.1449

Data Index: 1815462 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5620. Min Act: -0.1381

Data Index: 1498095 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5706. Min Act: -0.1050

Data Index: 1091535 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5685. Min Act: -0.1643

Data Index: 2012237 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5680. Min Act: -0.1304

Data Index: 2452407 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5619. Min Act: -0.1274

Data Index: 8212790 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5687. Min Act: -0.1419

Data Index: 5010528 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5611. Min Act: -0.1395

Data Index: 5703483 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 1.6439. Min Range: -1.6439

Max Act: 1.5650. Min Act: -0.1597

Data Index: 3808215 (Open Web Text)

Max Activating Token Index: 1023