Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1785 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.4445. Min Act: -0.1700

Data Index: 1148532 (Open Web Text)

Max Activating Token Index: 690

Text #1

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.3349. Min Act: -0.1700

Data Index: 5863417 (Open Web Text)

Max Activating Token Index: 795

Text #2

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.2607. Min Act: -0.1700

Data Index: 723772 (Open Web Text)

Max Activating Token Index: 43

Text #3

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.2759. Min Act: -0.1700

Data Index: 4819823 (Open Web Text)

Max Activating Token Index: 84

Text #4

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.2062. Min Act: -0.1700

Data Index: 6778586 (Open Web Text)

Max Activating Token Index: 384

Text #5

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.2041. Min Act: -0.1700

Data Index: 1315761 (Open Web Text)

Max Activating Token Index: 40

Text #6

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.1526. Min Act: -0.1700

Data Index: 3656843 (Open Web Text)

Max Activating Token Index: 401

Text #7

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.1581. Min Act: -0.1700

Data Index: 1611191 (Open Web Text)

Max Activating Token Index: 259

Text #8

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.1030. Min Act: -0.1700

Data Index: 1416926 (Open Web Text)

Max Activating Token Index: 180

Text #9

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0926. Min Act: -0.1700

Data Index: 5895508 (Open Web Text)

Max Activating Token Index: 216

Text #10

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.1087. Min Act: -0.1700

Data Index: 4299573 (Open Web Text)

Max Activating Token Index: 797

Text #11

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0448. Min Act: -0.1700

Data Index: 5713401 (Open Web Text)

Max Activating Token Index: 805

Text #12

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0868. Min Act: -0.1700

Data Index: 3917540 (Open Web Text)

Max Activating Token Index: 71

Text #13

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0393. Min Act: -0.1700

Data Index: 3665592 (Open Web Text)

Max Activating Token Index: 87

Text #14

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0050. Min Act: -0.1700

Data Index: 4768074 (Open Web Text)

Max Activating Token Index: 31

Text #15

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0207. Min Act: -0.1700

Data Index: 5300048 (Open Web Text)

Max Activating Token Index: 56

Text #16

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0065. Min Act: -0.1700

Data Index: 4202989 (Open Web Text)

Max Activating Token Index: 21

Text #17

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0036. Min Act: -0.1700

Data Index: 79122 (Open Web Text)

Max Activating Token Index: 952

Text #18

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0177. Min Act: -0.1700

Data Index: 5380016 (Open Web Text)

Max Activating Token Index: 690

Text #19

Max Range: 4.4445. Min Range: -4.4445

Max Act: 4.0074. Min Act: -0.1700

Data Index: 2334836 (Open Web Text)

Max Activating Token Index: 258