Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 748 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5800. Min Act: -0.1693

Data Index: 5709219 (Open Web Text)

Max Activating Token Index: 1015

Text #1

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5766. Min Act: -0.1543

Data Index: 6204751 (Open Web Text)

Max Activating Token Index: 916

Text #2

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5549. Min Act: -0.1684

Data Index: 1025704 (Open Web Text)

Max Activating Token Index: 96

Text #3

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5615. Min Act: -0.1535

Data Index: 1259127 (Open Web Text)

Max Activating Token Index: 40

Text #4

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5634. Min Act: -0.1688

Data Index: 8808473 (Open Web Text)

Max Activating Token Index: 25

Text #5

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5577. Min Act: -0.1601

Data Index: 1823304 (Open Web Text)

Max Activating Token Index: 98

Text #6

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5570. Min Act: -0.1698

Data Index: 439916 (Open Web Text)

Max Activating Token Index: 172

Text #7

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5551. Min Act: -0.1307

Data Index: 909329 (Open Web Text)

Max Activating Token Index: 78

Text #8

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5491. Min Act: -0.1644

Data Index: 840411 (Open Web Text)

Max Activating Token Index: 932

Text #9

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5625. Min Act: -0.1677

Data Index: 5821147 (Open Web Text)

Max Activating Token Index: 169

Text #10

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5504. Min Act: -0.1693

Data Index: 8314238 (Open Web Text)

Max Activating Token Index: 59

Text #11

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5521. Min Act: -0.1586

Data Index: 7652294 (Open Web Text)

Max Activating Token Index: 243

Text #12

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5460. Min Act: -0.1699

Data Index: 7652286 (Open Web Text)

Max Activating Token Index: 259

Text #13

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5355. Min Act: -0.1547

Data Index: 4799352 (Open Web Text)

Max Activating Token Index: 104

Text #14

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5433. Min Act: -0.1592

Data Index: 3210744 (Open Web Text)

Max Activating Token Index: 98

Text #15

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5335. Min Act: -0.1618

Data Index: 169269 (Open Web Text)

Max Activating Token Index: 62

Text #16

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5296. Min Act: -0.1700

Data Index: 188988 (Open Web Text)

Max Activating Token Index: 75

Text #17

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5283. Min Act: -0.1612

Data Index: 7501219 (Open Web Text)

Max Activating Token Index: 567

Text #18

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5252. Min Act: -0.1666

Data Index: 1203459 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 1.5800. Min Range: -1.5800

Max Act: 1.5280. Min Act: -0.1489

Data Index: 7574896 (Open Web Text)

Max Activating Token Index: 99