Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2520 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.2254. Min Act: -0.1700

Data Index: 789380 (Open Web Text)

Max Activating Token Index: 49

Text #1

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.2217. Min Act: -0.1700

Data Index: 6897886 (Open Web Text)

Max Activating Token Index: 352

Text #2

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.1777. Min Act: -0.1700

Data Index: 3463533 (Open Web Text)

Max Activating Token Index: 120

Text #3

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.1466. Min Act: -0.1700

Data Index: 3411356 (Open Web Text)

Max Activating Token Index: 533

Text #4

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.1162. Min Act: -0.1700

Data Index: 6639645 (Open Web Text)

Max Activating Token Index: 550

Text #5

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0964. Min Act: -0.1700

Data Index: 5015543 (Open Web Text)

Max Activating Token Index: 215

Text #6

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0843. Min Act: -0.1700

Data Index: 1898583 (Open Web Text)

Max Activating Token Index: 32

Text #7

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0958. Min Act: -0.1700

Data Index: 4753112 (Open Web Text)

Max Activating Token Index: 373

Text #8

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0894. Min Act: -0.1700

Data Index: 1062831 (Open Web Text)

Max Activating Token Index: 200

Text #9

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0697. Min Act: -0.1700

Data Index: 7260432 (Open Web Text)

Max Activating Token Index: 295

Text #10

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0744. Min Act: -0.1700

Data Index: 2939613 (Open Web Text)

Max Activating Token Index: 118

Text #11

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0667. Min Act: -0.1700

Data Index: 4416549 (Open Web Text)

Max Activating Token Index: 203

Text #12

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0687. Min Act: -0.1700

Data Index: 1087497 (Open Web Text)

Max Activating Token Index: 488

Text #13

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0688. Min Act: -0.1700

Data Index: 8812564 (Open Web Text)

Max Activating Token Index: 246

Text #14

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0984. Min Act: -0.1700

Data Index: 2848672 (Open Web Text)

Max Activating Token Index: 116

Text #15

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0569. Min Act: -0.1700

Data Index: 8537032 (Open Web Text)

Max Activating Token Index: 403

Text #16

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0766. Min Act: -0.1700

Data Index: 5061195 (Open Web Text)

Max Activating Token Index: 480

Text #17

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0828. Min Act: -0.1700

Data Index: 786710 (Open Web Text)

Max Activating Token Index: 172

Text #18

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0309. Min Act: -0.1700

Data Index: 4541649 (Open Web Text)

Max Activating Token Index: 277

Text #19

Max Range: 5.2254. Min Range: -5.2254

Max Act: 5.0409. Min Act: -0.1700

Data Index: 2866892 (Open Web Text)

Max Activating Token Index: 620