Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1510 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.9931. Min Act: -0.1700

Data Index: 7420329 (Open Web Text)

Max Activating Token Index: 822

Text #1

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.9926. Min Act: -0.1700

Data Index: 8704527 (Open Web Text)

Max Activating Token Index: 708

Text #2

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.9878. Min Act: -0.1700

Data Index: 6404910 (Open Web Text)

Max Activating Token Index: 655

Text #3

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.9177. Min Act: -0.1700

Data Index: 7644376 (Open Web Text)

Max Activating Token Index: 741

Text #4

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.9404. Min Act: -0.1700

Data Index: 5145481 (Open Web Text)

Max Activating Token Index: 749

Text #5

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.8155. Min Act: -0.1700

Data Index: 3634996 (Open Web Text)

Max Activating Token Index: 123

Text #6

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.8306. Min Act: -0.1700

Data Index: 3888995 (Open Web Text)

Max Activating Token Index: 737

Text #7

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.8196. Min Act: -0.1700

Data Index: 3575667 (Open Web Text)

Max Activating Token Index: 790

Text #8

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.8113. Min Act: -0.1700

Data Index: 1207803 (Open Web Text)

Max Activating Token Index: 720

Text #9

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7839. Min Act: -0.1700

Data Index: 6021309 (Open Web Text)

Max Activating Token Index: 690

Text #10

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7471. Min Act: -0.1700

Data Index: 5690325 (Open Web Text)

Max Activating Token Index: 101

Text #11

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.8118. Min Act: -0.1700

Data Index: 6832443 (Open Web Text)

Max Activating Token Index: 856

Text #12

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7290. Min Act: -0.1700

Data Index: 7377359 (Open Web Text)

Max Activating Token Index: 531

Text #13

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7542. Min Act: -0.1700

Data Index: 2816154 (Open Web Text)

Max Activating Token Index: 705

Text #14

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7369. Min Act: -0.1700

Data Index: 5064278 (Open Web Text)

Max Activating Token Index: 263

Text #15

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7285. Min Act: -0.1700

Data Index: 6328753 (Open Web Text)

Max Activating Token Index: 988

Text #16

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7469. Min Act: -0.1700

Data Index: 7296396 (Open Web Text)

Max Activating Token Index: 407

Text #17

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7135. Min Act: -0.1700

Data Index: 4112515 (Open Web Text)

Max Activating Token Index: 388

Text #18

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7430. Min Act: -0.1700

Data Index: 670158 (Open Web Text)

Max Activating Token Index: 518

Text #19

Max Range: 4.9931. Min Range: -4.9931

Max Act: 4.7189. Min Act: -0.1700

Data Index: 5314953 (Open Web Text)

Max Activating Token Index: 201