Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 614 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.6735. Min Act: -0.1059

Data Index: 8668805 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.6600. Min Act: -0.1102

Data Index: 6650207 (Open Web Text)

Max Activating Token Index: 762

Text #2

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.5234. Min Act: -0.1622

Data Index: 7427898 (Open Web Text)

Max Activating Token Index: 9

Text #3

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.5109. Min Act: -0.0190

Data Index: 345149 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.5216. Min Act: 0.2922

Data Index: 1245269 (Open Web Text)

Max Activating Token Index: 837

Text #5

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.4560. Min Act: -0.0310

Data Index: 2780744 (Open Web Text)

Max Activating Token Index: 28

Text #6

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.4127. Min Act: -0.1416

Data Index: 6470895 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.4339. Min Act: 0.5329

Data Index: 3074217 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3943. Min Act: -0.1614

Data Index: 2278398 (Open Web Text)

Max Activating Token Index: 26

Text #9

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3489. Min Act: -0.1525

Data Index: 6684996 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3607. Min Act: -0.1429

Data Index: 8748856 (Open Web Text)

Max Activating Token Index: 15

Text #11

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3476. Min Act: -0.1082

Data Index: 583710 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3426. Min Act: -0.0204

Data Index: 7477875 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3533. Min Act: -0.0663

Data Index: 583702 (Open Web Text)

Max Activating Token Index: 15

Text #14

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3248. Min Act: -0.0727

Data Index: 961760 (Open Web Text)

Max Activating Token Index: 835

Text #15

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3150. Min Act: -0.1685

Data Index: 1119328 (Open Web Text)

Max Activating Token Index: 19

Text #16

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.3173. Min Act: -0.1693

Data Index: 7701174 (Open Web Text)

Max Activating Token Index: 249

Text #17

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.2964. Min Act: -0.0182

Data Index: 2234697 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.2910. Min Act: -0.1636

Data Index: 487586 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 4.6735. Min Range: -4.6735

Max Act: 4.2407. Min Act: -0.1083

Data Index: 3256997 (Open Web Text)

Max Activating Token Index: 191