Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 350 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.5114. Min Act: -0.1700

Data Index: 6251651 (Open Web Text)

Max Activating Token Index: 413

Text #1

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4549. Min Act: -0.1700

Data Index: 3224348 (Open Web Text)

Max Activating Token Index: 626

Text #2

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4515. Min Act: -0.1700

Data Index: 5506332 (Open Web Text)

Max Activating Token Index: 807

Text #3

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4372. Min Act: -0.1700

Data Index: 3423537 (Open Web Text)

Max Activating Token Index: 19

Text #4

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4120. Min Act: -0.1700

Data Index: 215231 (Open Web Text)

Max Activating Token Index: 598

Text #5

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4001. Min Act: -0.1700

Data Index: 8771496 (Open Web Text)

Max Activating Token Index: 960

Text #6

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4282. Min Act: -0.1700

Data Index: 3646587 (Open Web Text)

Max Activating Token Index: 15

Text #7

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4368. Min Act: -0.1700

Data Index: 561681 (Open Web Text)

Max Activating Token Index: 330

Text #8

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4051. Min Act: -0.1700

Data Index: 7234553 (Open Web Text)

Max Activating Token Index: 867

Text #9

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4064. Min Act: -0.1700

Data Index: 7989405 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.4188. Min Act: -0.1700

Data Index: 5400970 (Open Web Text)

Max Activating Token Index: 344

Text #11

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3615. Min Act: -0.1700

Data Index: 5063049 (Open Web Text)

Max Activating Token Index: 330

Text #12

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3967. Min Act: -0.1700

Data Index: 3600990 (Open Web Text)

Max Activating Token Index: 199

Text #13

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3922. Min Act: -0.1700

Data Index: 8086059 (Open Web Text)

Max Activating Token Index: 255

Text #14

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3097. Min Act: -0.1700

Data Index: 3245701 (Open Web Text)

Max Activating Token Index: 491

Text #15

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3410. Min Act: -0.1700

Data Index: 6465266 (Open Web Text)

Max Activating Token Index: 13

Text #16

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3725. Min Act: -0.1700

Data Index: 2890153 (Open Web Text)

Max Activating Token Index: 37

Text #17

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3523. Min Act: -0.1700

Data Index: 1802857 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3415. Min Act: -0.1700

Data Index: 5496816 (Open Web Text)

Max Activating Token Index: 12

Text #19

Max Range: 2.5114. Min Range: -2.5114

Max Act: 2.3433. Min Act: -0.1700

Data Index: 7214481 (Open Web Text)

Max Activating Token Index: 7