Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2021 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.9029. Min Act: -0.1700

Data Index: 6244990 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8697. Min Act: -0.1700

Data Index: 3421371 (Open Web Text)

Max Activating Token Index: 11

Text #2

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8791. Min Act: -0.1700

Data Index: 284598 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8715. Min Act: -0.1700

Data Index: 7487350 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8809. Min Act: -0.1700

Data Index: 8345757 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8629. Min Act: -0.1700

Data Index: 6074775 (Open Web Text)

Max Activating Token Index: 11

Text #6

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8515. Min Act: -0.1700

Data Index: 907872 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8597. Min Act: -0.1700

Data Index: 8689713 (Open Web Text)

Max Activating Token Index: 11

Text #8

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8506. Min Act: -0.1700

Data Index: 4850066 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8376. Min Act: -0.1700

Data Index: 8291481 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8469. Min Act: -0.1700

Data Index: 5881691 (Open Web Text)

Max Activating Token Index: 23

Text #11

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8440. Min Act: -0.1700

Data Index: 143010 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8358. Min Act: -0.1700

Data Index: 1662895 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8478. Min Act: -0.1700

Data Index: 2562409 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8505. Min Act: -0.1700

Data Index: 5930247 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8217. Min Act: -0.1700

Data Index: 5402414 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8299. Min Act: -0.1700

Data Index: 1171929 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8262. Min Act: -0.1700

Data Index: 3305383 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8234. Min Act: -0.1700

Data Index: 2031972 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 2.9029. Min Range: -2.9029

Max Act: 2.8234. Min Act: -0.1700

Data Index: 2712004 (Open Web Text)

Max Activating Token Index: 2