Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1140 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.3238. Min Act: -0.1700

Data Index: 7903654 (Open Web Text)

Max Activating Token Index: 929

Text #1

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.2501. Min Act: -0.1700

Data Index: 2372180 (Open Web Text)

Max Activating Token Index: 576

Text #2

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.2142. Min Act: -0.1700

Data Index: 662904 (Open Web Text)

Max Activating Token Index: 595

Text #3

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.2190. Min Act: -0.1700

Data Index: 2584641 (Open Web Text)

Max Activating Token Index: 111

Text #4

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.2226. Min Act: -0.1700

Data Index: 2927548 (Open Web Text)

Max Activating Token Index: 146

Text #5

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.2063. Min Act: -0.1700

Data Index: 4464071 (Open Web Text)

Max Activating Token Index: 205

Text #6

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1734. Min Act: -0.1700

Data Index: 2887187 (Open Web Text)

Max Activating Token Index: 684

Text #7

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1965. Min Act: -0.1700

Data Index: 4372072 (Open Web Text)

Max Activating Token Index: 397

Text #8

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1829. Min Act: -0.1700

Data Index: 918324 (Open Web Text)

Max Activating Token Index: 580

Text #9

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1768. Min Act: -0.1700

Data Index: 6149505 (Open Web Text)

Max Activating Token Index: 161

Text #10

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1713. Min Act: -0.1700

Data Index: 7913865 (Open Web Text)

Max Activating Token Index: 642

Text #11

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1653. Min Act: -0.1700

Data Index: 7326508 (Open Web Text)

Max Activating Token Index: 203

Text #12

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1604. Min Act: -0.1700

Data Index: 2644148 (Open Web Text)

Max Activating Token Index: 660

Text #13

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1006. Min Act: -0.1700

Data Index: 1942305 (Open Web Text)

Max Activating Token Index: 997

Text #14

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1115. Min Act: -0.1700

Data Index: 6261758 (Open Web Text)

Max Activating Token Index: 742

Text #15

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1137. Min Act: -0.1700

Data Index: 309723 (Open Web Text)

Max Activating Token Index: 626

Text #16

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.0996. Min Act: -0.1700

Data Index: 2399943 (Open Web Text)

Max Activating Token Index: 209

Text #17

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1183. Min Act: -0.1700

Data Index: 84399 (Open Web Text)

Max Activating Token Index: 566

Text #18

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1109. Min Act: -0.1700

Data Index: 4992928 (Open Web Text)

Max Activating Token Index: 498

Text #19

Max Range: 4.3238. Min Range: -4.3238

Max Act: 4.1196. Min Act: -0.1700

Data Index: 6325689 (Open Web Text)

Max Activating Token Index: 73