Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 955 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.7530. Min Act: -0.1700

Data Index: 8617230 (Open Web Text)

Max Activating Token Index: 25

Text #1

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6958. Min Act: -0.1700

Data Index: 6227073 (Open Web Text)

Max Activating Token Index: 726

Text #2

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6452. Min Act: -0.1700

Data Index: 7177614 (Open Web Text)

Max Activating Token Index: 93

Text #3

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6648. Min Act: -0.1700

Data Index: 4371662 (Open Web Text)

Max Activating Token Index: 446

Text #4

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6448. Min Act: -0.1700

Data Index: 4655718 (Open Web Text)

Max Activating Token Index: 763

Text #5

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6087. Min Act: -0.1700

Data Index: 4530945 (Open Web Text)

Max Activating Token Index: 761

Text #6

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5841. Min Act: -0.1700

Data Index: 5722543 (Open Web Text)

Max Activating Token Index: 311

Text #7

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6375. Min Act: -0.1700

Data Index: 61080 (Open Web Text)

Max Activating Token Index: 118

Text #8

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6215. Min Act: -0.1700

Data Index: 4850760 (Open Web Text)

Max Activating Token Index: 409

Text #9

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6056. Min Act: -0.1700

Data Index: 7753684 (Open Web Text)

Max Activating Token Index: 461

Text #10

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5882. Min Act: -0.1700

Data Index: 6228749 (Open Web Text)

Max Activating Token Index: 740

Text #11

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5763. Min Act: -0.1700

Data Index: 5262580 (Open Web Text)

Max Activating Token Index: 926

Text #12

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5843. Min Act: -0.1700

Data Index: 264590 (Open Web Text)

Max Activating Token Index: 971

Text #13

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5675. Min Act: -0.1700

Data Index: 489438 (Open Web Text)

Max Activating Token Index: 484

Text #14

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6071. Min Act: -0.1700

Data Index: 5602567 (Open Web Text)

Max Activating Token Index: 87

Text #15

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5707. Min Act: -0.1700

Data Index: 1771572 (Open Web Text)

Max Activating Token Index: 202

Text #16

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6048. Min Act: -0.1700

Data Index: 4828603 (Open Web Text)

Max Activating Token Index: 78

Text #17

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.6142. Min Act: -0.1700

Data Index: 1800764 (Open Web Text)

Max Activating Token Index: 111

Text #18

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5619. Min Act: -0.1700

Data Index: 1904680 (Open Web Text)

Max Activating Token Index: 448

Text #19

Max Range: 4.7530. Min Range: -4.7530

Max Act: 4.5805. Min Act: -0.1700

Data Index: 2118970 (Open Web Text)

Max Activating Token Index: 12