Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1933 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4294. Min Range: -4.4294

Max Act: 4.4294. Min Act: -0.0935

Data Index: 11427 (Open Web Text)

Max Activating Token Index: 29

Text #1

Max Range: 4.4294. Min Range: -4.4294

Max Act: 4.0529. Min Act: -0.1631

Data Index: 8172655 (Open Web Text)

Max Activating Token Index: 253

Text #2

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.9857. Min Act: -0.1366

Data Index: 11428 (Open Web Text)

Max Activating Token Index: 138

Text #3

Max Range: 4.4294. Min Range: -4.4294

Max Act: 4.0070. Min Act: -0.1419

Data Index: 8695926 (Open Web Text)

Max Activating Token Index: 486

Text #4

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.7576. Min Act: -0.1070

Data Index: 645661 (Open Web Text)

Max Activating Token Index: 916

Text #5

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.6249. Min Act: -0.1341

Data Index: 1829509 (Open Web Text)

Max Activating Token Index: 777

Text #6

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.5928. Min Act: -0.1422

Data Index: 1076601 (Open Web Text)

Max Activating Token Index: 498

Text #7

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.4665. Min Act: -0.1504

Data Index: 7540473 (Open Web Text)

Max Activating Token Index: 494

Text #8

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.4222. Min Act: -0.1127

Data Index: 5560273 (Open Web Text)

Max Activating Token Index: 343

Text #9

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.3254. Min Act: -0.1377

Data Index: 7966212 (Open Web Text)

Max Activating Token Index: 277

Text #10

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.3231. Min Act: -0.1345

Data Index: 5410553 (Open Web Text)

Max Activating Token Index: 655

Text #11

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.2642. Min Act: -0.0689

Data Index: 4122220 (Open Web Text)

Max Activating Token Index: 224

Text #12

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.2362. Min Act: -0.1267

Data Index: 1829510 (Open Web Text)

Max Activating Token Index: 204

Text #13

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.2156. Min Act: -0.1470

Data Index: 481744 (Open Web Text)

Max Activating Token Index: 829

Text #14

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.2511. Min Act: -0.1489

Data Index: 4159496 (Open Web Text)

Max Activating Token Index: 331

Text #15

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.2349. Min Act: -0.1369

Data Index: 7413992 (Open Web Text)

Max Activating Token Index: 566

Text #16

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.1989. Min Act: -0.1078

Data Index: 6234021 (Open Web Text)

Max Activating Token Index: 308

Text #17

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.1905. Min Act: -0.1506

Data Index: 1701531 (Open Web Text)

Max Activating Token Index: 577

Text #18

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.1766. Min Act: -0.1470

Data Index: 4387039 (Open Web Text)

Max Activating Token Index: 689

Text #19

Max Range: 4.4294. Min Range: -4.4294

Max Act: 3.1410. Min Act: -0.1632

Data Index: 1083232 (Open Web Text)

Max Activating Token Index: 757