Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1289 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8957. Min Act: -0.1700

Data Index: 7342108 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8614. Min Act: -0.1700

Data Index: 2662364 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8544. Min Act: -0.1700

Data Index: 5969800 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8482. Min Act: -0.1700

Data Index: 44241 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8540. Min Act: -0.1700

Data Index: 5116079 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8395. Min Act: -0.1700

Data Index: 2714268 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8472. Min Act: -0.1700

Data Index: 3317410 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8346. Min Act: -0.1700

Data Index: 6228172 (Open Web Text)

Max Activating Token Index: 8

Text #8

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8377. Min Act: -0.1700

Data Index: 4970971 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8192. Min Act: -0.1700

Data Index: 7788790 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8246. Min Act: -0.1700

Data Index: 8332118 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8304. Min Act: -0.1700

Data Index: 6930463 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8270. Min Act: -0.1700

Data Index: 606259 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8309. Min Act: -0.1700

Data Index: 8659409 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8279. Min Act: -0.1700

Data Index: 3025797 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8087. Min Act: -0.1700

Data Index: 2146697 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8055. Min Act: -0.1700

Data Index: 5328028 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8069. Min Act: -0.1700

Data Index: 3755178 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8047. Min Act: -0.1700

Data Index: 2585649 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 2.8957. Min Range: -2.8957

Max Act: 2.8042. Min Act: -0.1700

Data Index: 3125221 (Open Web Text)

Max Activating Token Index: 4