Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1820 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1520. Min Range: -5.1520

Max Act: 5.1520. Min Act: -0.1700

Data Index: 8695858 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 5.1520. Min Range: -5.1520

Max Act: 5.0654. Min Act: -0.1700

Data Index: 2877807 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 5.1520. Min Range: -5.1520

Max Act: 5.0405. Min Act: -0.1700

Data Index: 1336161 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 5.1520. Min Range: -5.1520

Max Act: 5.0379. Min Act: -0.1700

Data Index: 82980 (Open Web Text)

Max Activating Token Index: 15

Text #4

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9910. Min Act: -0.1700

Data Index: 7950406 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9915. Min Act: -0.1700

Data Index: 3539842 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 5.1520. Min Range: -5.1520

Max Act: 5.0080. Min Act: -0.1700

Data Index: 4213008 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9627. Min Act: -0.1700

Data Index: 8603721 (Open Web Text)

Max Activating Token Index: 10

Text #8

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9746. Min Act: -0.1700

Data Index: 3423136 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9741. Min Act: -0.1700

Data Index: 440990 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9279. Min Act: -0.1700

Data Index: 4637087 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9284. Min Act: -0.1700

Data Index: 998366 (Open Web Text)

Max Activating Token Index: 8

Text #12

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9508. Min Act: -0.1700

Data Index: 4233665 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9110. Min Act: -0.1700

Data Index: 1275655 (Open Web Text)

Max Activating Token Index: 11

Text #14

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9029. Min Act: -0.1700

Data Index: 4880792 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9072. Min Act: -0.1700

Data Index: 4967608 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9082. Min Act: -0.1700

Data Index: 528367 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.8926. Min Act: -0.1700

Data Index: 5507553 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9212. Min Act: -0.1700

Data Index: 6174227 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.1520. Min Range: -5.1520

Max Act: 4.9184. Min Act: -0.1700

Data Index: 3202108 (Open Web Text)

Max Activating Token Index: 4