Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 926 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0429. Min Range: -3.0429

Max Act: 3.0429. Min Act: -0.1700

Data Index: 7974962 (Open Web Text)

Max Activating Token Index: 1009

Text #1

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.9340. Min Act: -0.1700

Data Index: 3382131 (Open Web Text)

Max Activating Token Index: 927

Text #2

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.8373. Min Act: -0.1700

Data Index: 2442102 (Open Web Text)

Max Activating Token Index: 798

Text #3

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.8687. Min Act: -0.1700

Data Index: 2446607 (Open Web Text)

Max Activating Token Index: 645

Text #4

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.8145. Min Act: -0.1700

Data Index: 3915364 (Open Web Text)

Max Activating Token Index: 530

Text #5

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.8188. Min Act: -0.1700

Data Index: 6496967 (Open Web Text)

Max Activating Token Index: 460

Text #6

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.7070. Min Act: -0.1700

Data Index: 8264834 (Open Web Text)

Max Activating Token Index: 656

Text #7

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.7269. Min Act: -0.1700

Data Index: 8323388 (Open Web Text)

Max Activating Token Index: 257

Text #8

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.7192. Min Act: -0.1700

Data Index: 1463815 (Open Web Text)

Max Activating Token Index: 511

Text #9

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.7052. Min Act: -0.1700

Data Index: 1587719 (Open Web Text)

Max Activating Token Index: 650

Text #10

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.7003. Min Act: -0.1700

Data Index: 7303932 (Open Web Text)

Max Activating Token Index: 636

Text #11

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6514. Min Act: -0.1700

Data Index: 4474219 (Open Web Text)

Max Activating Token Index: 922

Text #12

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6706. Min Act: -0.1700

Data Index: 1392509 (Open Web Text)

Max Activating Token Index: 339

Text #13

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6594. Min Act: -0.1700

Data Index: 2445920 (Open Web Text)

Max Activating Token Index: 240

Text #14

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6608. Min Act: -0.1700

Data Index: 8815619 (Open Web Text)

Max Activating Token Index: 960

Text #15

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6471. Min Act: -0.1700

Data Index: 7595764 (Open Web Text)

Max Activating Token Index: 501

Text #16

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6591. Min Act: -0.1700

Data Index: 4164042 (Open Web Text)

Max Activating Token Index: 862

Text #17

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6210. Min Act: -0.1700

Data Index: 2013880 (Open Web Text)

Max Activating Token Index: 877

Text #18

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6257. Min Act: -0.1700

Data Index: 1988673 (Open Web Text)

Max Activating Token Index: 118

Text #19

Max Range: 3.0429. Min Range: -3.0429

Max Act: 2.6092. Min Act: -0.1700

Data Index: 8073576 (Open Web Text)

Max Activating Token Index: 473