Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 814 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.5553. Min Act: -0.1700

Data Index: 2184651 (Open Web Text)

Max Activating Token Index: 552

Text #1

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.5541. Min Act: -0.1700

Data Index: 1925693 (Open Web Text)

Max Activating Token Index: 585

Text #2

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.5334. Min Act: -0.1700

Data Index: 7480676 (Open Web Text)

Max Activating Token Index: 484

Text #3

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.5111. Min Act: -0.1700

Data Index: 7480677 (Open Web Text)

Max Activating Token Index: 634

Text #4

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4899. Min Act: -0.1700

Data Index: 4812079 (Open Web Text)

Max Activating Token Index: 784

Text #5

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4724. Min Act: -0.1700

Data Index: 3608632 (Open Web Text)

Max Activating Token Index: 171

Text #6

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4918. Min Act: -0.1700

Data Index: 7480680 (Open Web Text)

Max Activating Token Index: 209

Text #7

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4653. Min Act: -0.1700

Data Index: 5707195 (Open Web Text)

Max Activating Token Index: 63

Text #8

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4716. Min Act: -0.1700

Data Index: 6024865 (Open Web Text)

Max Activating Token Index: 306

Text #9

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4667. Min Act: -0.1700

Data Index: 468204 (Open Web Text)

Max Activating Token Index: 732

Text #10

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4722. Min Act: -0.1700

Data Index: 7480678 (Open Web Text)

Max Activating Token Index: 848

Text #11

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4761. Min Act: -0.1700

Data Index: 7480675 (Open Web Text)

Max Activating Token Index: 261

Text #12

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4372. Min Act: -0.1700

Data Index: 5647298 (Open Web Text)

Max Activating Token Index: 896

Text #13

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4219. Min Act: -0.1700

Data Index: 3996515 (Open Web Text)

Max Activating Token Index: 838

Text #14

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4563. Min Act: -0.1700

Data Index: 8653553 (Open Web Text)

Max Activating Token Index: 143

Text #15

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4606. Min Act: -0.1700

Data Index: 4922863 (Open Web Text)

Max Activating Token Index: 126

Text #16

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4417. Min Act: -0.1700

Data Index: 1394075 (Open Web Text)

Max Activating Token Index: 489

Text #17

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4390. Min Act: -0.1700

Data Index: 7310876 (Open Web Text)

Max Activating Token Index: 117

Text #18

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4225. Min Act: -0.1700

Data Index: 6778101 (Open Web Text)

Max Activating Token Index: 193

Text #19

Max Range: 3.5553. Min Range: -3.5553

Max Act: 3.4317. Min Act: -0.1700

Data Index: 149220 (Open Web Text)

Max Activating Token Index: 949