Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1441 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.9282. Min Act: -0.1700

Data Index: 6939912 (Open Web Text)

Max Activating Token Index: 683

Text #1

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8844. Min Act: -0.1700

Data Index: 800869 (Open Web Text)

Max Activating Token Index: 26

Text #2

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8985. Min Act: -0.1700

Data Index: 6777861 (Open Web Text)

Max Activating Token Index: 652

Text #3

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8939. Min Act: -0.1700

Data Index: 8007705 (Open Web Text)

Max Activating Token Index: 674

Text #4

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.9010. Min Act: -0.1700

Data Index: 1488827 (Open Web Text)

Max Activating Token Index: 424

Text #5

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8674. Min Act: -0.1700

Data Index: 7280540 (Open Web Text)

Max Activating Token Index: 434

Text #6

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8506. Min Act: -0.1700

Data Index: 4539327 (Open Web Text)

Max Activating Token Index: 115

Text #7

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8673. Min Act: -0.1700

Data Index: 4269757 (Open Web Text)

Max Activating Token Index: 717

Text #8

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8512. Min Act: -0.1700

Data Index: 7048926 (Open Web Text)

Max Activating Token Index: 849

Text #9

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8647. Min Act: -0.1700

Data Index: 1015567 (Open Web Text)

Max Activating Token Index: 431

Text #10

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8140. Min Act: -0.1700

Data Index: 7159178 (Open Web Text)

Max Activating Token Index: 565

Text #11

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8234. Min Act: -0.1700

Data Index: 6109650 (Open Web Text)

Max Activating Token Index: 911

Text #12

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8359. Min Act: -0.1700

Data Index: 7552801 (Open Web Text)

Max Activating Token Index: 87

Text #13

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8192. Min Act: -0.1700

Data Index: 7781604 (Open Web Text)

Max Activating Token Index: 675

Text #14

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8340. Min Act: -0.1700

Data Index: 2652218 (Open Web Text)

Max Activating Token Index: 346

Text #15

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8187. Min Act: -0.1700

Data Index: 6547567 (Open Web Text)

Max Activating Token Index: 378

Text #16

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.8197. Min Act: -0.1700

Data Index: 5770782 (Open Web Text)

Max Activating Token Index: 70

Text #17

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.7890. Min Act: -0.1700

Data Index: 2238334 (Open Web Text)

Max Activating Token Index: 166

Text #18

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.7973. Min Act: -0.1700

Data Index: 4674530 (Open Web Text)

Max Activating Token Index: 590

Text #19

Max Range: 3.9282. Min Range: -3.9282

Max Act: 3.7994. Min Act: -0.1700

Data Index: 1337771 (Open Web Text)

Max Activating Token Index: 704