Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 882 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.5086. Min Act: -0.1700

Data Index: 1056894 (Open Web Text)

Max Activating Token Index: 649

Text #1

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4969. Min Act: -0.1700

Data Index: 4442142 (Open Web Text)

Max Activating Token Index: 93

Text #2

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.5036. Min Act: -0.1700

Data Index: 3684306 (Open Web Text)

Max Activating Token Index: 51

Text #3

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4803. Min Act: -0.1700

Data Index: 912589 (Open Web Text)

Max Activating Token Index: 12

Text #4

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4882. Min Act: -0.1698

Data Index: 2230553 (Open Web Text)

Max Activating Token Index: 698

Text #5

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4801. Min Act: -0.1682

Data Index: 6458966 (Open Web Text)

Max Activating Token Index: 154

Text #6

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4829. Min Act: -0.1700

Data Index: 2903216 (Open Web Text)

Max Activating Token Index: 111

Text #7

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4656. Min Act: -0.1700

Data Index: 5154566 (Open Web Text)

Max Activating Token Index: 588

Text #8

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4697. Min Act: -0.1700

Data Index: 3846415 (Open Web Text)

Max Activating Token Index: 877

Text #9

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4653. Min Act: -0.1700

Data Index: 5208885 (Open Web Text)

Max Activating Token Index: 816

Text #10

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4720. Min Act: -0.1700

Data Index: 1505346 (Open Web Text)

Max Activating Token Index: 524

Text #11

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4530. Min Act: -0.1700

Data Index: 3150883 (Open Web Text)

Max Activating Token Index: 405

Text #12

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4587. Min Act: -0.1700

Data Index: 5407184 (Open Web Text)

Max Activating Token Index: 55

Text #13

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4491. Min Act: -0.1673

Data Index: 5820924 (Open Web Text)

Max Activating Token Index: 32

Text #14

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4508. Min Act: -0.1697

Data Index: 6695576 (Open Web Text)

Max Activating Token Index: 502

Text #15

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4442. Min Act: -0.1700

Data Index: 5037951 (Open Web Text)

Max Activating Token Index: 543

Text #16

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4394. Min Act: -0.1700

Data Index: 1867439 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4500. Min Act: -0.1700

Data Index: 4671983 (Open Web Text)

Max Activating Token Index: 763

Text #18

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4351. Min Act: -0.1700

Data Index: 205380 (Open Web Text)

Max Activating Token Index: 161

Text #19

Max Range: 1.5086. Min Range: -1.5086

Max Act: 1.4348. Min Act: -0.1700

Data Index: 3223679 (Open Web Text)

Max Activating Token Index: 635