Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1125 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4245. Min Act: -0.1370

Data Index: 7027432 (Open Web Text)

Max Activating Token Index: 114

Text #1

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4211. Min Act: -0.1454

Data Index: 8482241 (Open Web Text)

Max Activating Token Index: 234

Text #2

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4191. Min Act: -0.1682

Data Index: 4604406 (Open Web Text)

Max Activating Token Index: 392

Text #3

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4154. Min Act: -0.1507

Data Index: 6715165 (Open Web Text)

Max Activating Token Index: 98

Text #4

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4118. Min Act: -0.1278

Data Index: 4816816 (Open Web Text)

Max Activating Token Index: 145

Text #5

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4116. Min Act: -0.1631

Data Index: 7958603 (Open Web Text)

Max Activating Token Index: 71

Text #6

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4060. Min Act: -0.0969

Data Index: 3499970 (Open Web Text)

Max Activating Token Index: 38

Text #7

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4028. Min Act: -0.1314

Data Index: 8798434 (Open Web Text)

Max Activating Token Index: 719

Text #8

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4001. Min Act: -0.1654

Data Index: 4151145 (Open Web Text)

Max Activating Token Index: 718

Text #9

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3968. Min Act: -0.1278

Data Index: 2782694 (Open Web Text)

Max Activating Token Index: 424

Text #10

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4028. Min Act: -0.1530

Data Index: 2272970 (Open Web Text)

Max Activating Token Index: 142

Text #11

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.4003. Min Act: -0.1320

Data Index: 3810812 (Open Web Text)

Max Activating Token Index: 266

Text #12

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3960. Min Act: -0.1691

Data Index: 3772144 (Open Web Text)

Max Activating Token Index: 592

Text #13

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3913. Min Act: -0.1501

Data Index: 7574487 (Open Web Text)

Max Activating Token Index: 115

Text #14

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3902. Min Act: -0.1606

Data Index: 8256443 (Open Web Text)

Max Activating Token Index: 155

Text #15

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3918. Min Act: -0.1364

Data Index: 1316728 (Open Web Text)

Max Activating Token Index: 174

Text #16

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3932. Min Act: -0.1684

Data Index: 1425555 (Open Web Text)

Max Activating Token Index: 447

Text #17

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3892. Min Act: -0.1092

Data Index: 1316723 (Open Web Text)

Max Activating Token Index: 118

Text #18

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3838. Min Act: -0.1257

Data Index: 1733878 (Open Web Text)

Max Activating Token Index: 97

Text #19

Max Range: 1.4245. Min Range: -1.4245

Max Act: 1.3870. Min Act: -0.1699

Data Index: 2210512 (Open Web Text)

Max Activating Token Index: 243