Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1835 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.8107. Min Act: -0.1700

Data Index: 264479 (Open Web Text)

Max Activating Token Index: 39

Text #1

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6914. Min Act: -0.1700

Data Index: 6943169 (Open Web Text)

Max Activating Token Index: 34

Text #2

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6970. Min Act: -0.1700

Data Index: 1466945 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6637. Min Act: -0.1700

Data Index: 7368654 (Open Web Text)

Max Activating Token Index: 736

Text #4

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6079. Min Act: -0.1700

Data Index: 521382 (Open Web Text)

Max Activating Token Index: 63

Text #5

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6079. Min Act: -0.1700

Data Index: 1379007 (Open Web Text)

Max Activating Token Index: 30

Text #6

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6052. Min Act: -0.1700

Data Index: 387967 (Open Web Text)

Max Activating Token Index: 483

Text #7

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5901. Min Act: -0.1700

Data Index: 5163403 (Open Web Text)

Max Activating Token Index: 57

Text #8

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5874. Min Act: -0.1700

Data Index: 8732384 (Open Web Text)

Max Activating Token Index: 472

Text #9

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.6044. Min Act: -0.1700

Data Index: 1546034 (Open Web Text)

Max Activating Token Index: 892

Text #10

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5752. Min Act: -0.1700

Data Index: 6892185 (Open Web Text)

Max Activating Token Index: 852

Text #11

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5624. Min Act: -0.1700

Data Index: 7957321 (Open Web Text)

Max Activating Token Index: 471

Text #12

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5395. Min Act: -0.1700

Data Index: 441823 (Open Web Text)

Max Activating Token Index: 191

Text #13

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5328. Min Act: -0.1700

Data Index: 6753017 (Open Web Text)

Max Activating Token Index: 98

Text #14

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5451. Min Act: -0.1700

Data Index: 8348274 (Open Web Text)

Max Activating Token Index: 528

Text #15

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5334. Min Act: -0.1700

Data Index: 2332143 (Open Web Text)

Max Activating Token Index: 992

Text #16

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5354. Min Act: -0.1700

Data Index: 1216942 (Open Web Text)

Max Activating Token Index: 116

Text #17

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5286. Min Act: -0.1700

Data Index: 3592343 (Open Web Text)

Max Activating Token Index: 330

Text #18

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5100. Min Act: -0.1700

Data Index: 3914163 (Open Web Text)

Max Activating Token Index: 491

Text #19

Max Range: 3.8107. Min Range: -3.8107

Max Act: 3.5191. Min Act: -0.1700

Data Index: 4467030 (Open Web Text)

Max Activating Token Index: 641