Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2256 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.9204. Min Act: -0.1700

Data Index: 462033 (Open Web Text)

Max Activating Token Index: 761

Text #1

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.8841. Min Act: -0.1700

Data Index: 640332 (Open Web Text)

Max Activating Token Index: 600

Text #2

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7817. Min Act: -0.1700

Data Index: 6301144 (Open Web Text)

Max Activating Token Index: 575

Text #3

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7559. Min Act: -0.1700

Data Index: 6854384 (Open Web Text)

Max Activating Token Index: 52

Text #4

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7827. Min Act: -0.1700

Data Index: 3053188 (Open Web Text)

Max Activating Token Index: 612

Text #5

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7612. Min Act: -0.1700

Data Index: 8528752 (Open Web Text)

Max Activating Token Index: 879

Text #6

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7740. Min Act: -0.1700

Data Index: 6839569 (Open Web Text)

Max Activating Token Index: 55

Text #7

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7476. Min Act: -0.1700

Data Index: 4403580 (Open Web Text)

Max Activating Token Index: 774

Text #8

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7560. Min Act: -0.1700

Data Index: 7906795 (Open Web Text)

Max Activating Token Index: 20

Text #9

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7591. Min Act: -0.1700

Data Index: 2743249 (Open Web Text)

Max Activating Token Index: 204

Text #10

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7551. Min Act: -0.1700

Data Index: 4948800 (Open Web Text)

Max Activating Token Index: 977

Text #11

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7342. Min Act: -0.1700

Data Index: 4831101 (Open Web Text)

Max Activating Token Index: 276

Text #12

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7608. Min Act: -0.1700

Data Index: 3067235 (Open Web Text)

Max Activating Token Index: 90

Text #13

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7415. Min Act: -0.1700

Data Index: 7993040 (Open Web Text)

Max Activating Token Index: 23

Text #14

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7656. Min Act: -0.1700

Data Index: 1593017 (Open Web Text)

Max Activating Token Index: 973

Text #15

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7812. Min Act: -0.1700

Data Index: 1278969 (Open Web Text)

Max Activating Token Index: 941

Text #16

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7119. Min Act: -0.1700

Data Index: 6890126 (Open Web Text)

Max Activating Token Index: 326

Text #17

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7131. Min Act: -0.1700

Data Index: 3330005 (Open Web Text)

Max Activating Token Index: 34

Text #18

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7230. Min Act: -0.1700

Data Index: 1177166 (Open Web Text)

Max Activating Token Index: 953

Text #19

Max Range: 3.9204. Min Range: -3.9204

Max Act: 3.7547. Min Act: -0.1700

Data Index: 1386826 (Open Web Text)

Max Activating Token Index: 1010