Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1871 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.6613. Min Act: -0.1700

Data Index: 4881881 (Open Web Text)

Max Activating Token Index: 169

Text #1

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.6161. Min Act: -0.1700

Data Index: 3978562 (Open Web Text)

Max Activating Token Index: 919

Text #2

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4661. Min Act: -0.1700

Data Index: 5356714 (Open Web Text)

Max Activating Token Index: 864

Text #3

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4583. Min Act: -0.1700

Data Index: 1223623 (Open Web Text)

Max Activating Token Index: 189

Text #4

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4445. Min Act: -0.1700

Data Index: 1554618 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4591. Min Act: -0.1700

Data Index: 4316896 (Open Web Text)

Max Activating Token Index: 608

Text #6

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4105. Min Act: -0.1700

Data Index: 5372237 (Open Web Text)

Max Activating Token Index: 857

Text #7

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4283. Min Act: -0.1700

Data Index: 2493368 (Open Web Text)

Max Activating Token Index: 8

Text #8

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.4192. Min Act: -0.1700

Data Index: 8143747 (Open Web Text)

Max Activating Token Index: 459

Text #9

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3937. Min Act: -0.1700

Data Index: 6613727 (Open Web Text)

Max Activating Token Index: 563

Text #10

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3853. Min Act: -0.1700

Data Index: 1633253 (Open Web Text)

Max Activating Token Index: 443

Text #11

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3626. Min Act: -0.1700

Data Index: 4751042 (Open Web Text)

Max Activating Token Index: 272

Text #12

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3610. Min Act: -0.1700

Data Index: 7044959 (Open Web Text)

Max Activating Token Index: 141

Text #13

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3367. Min Act: -0.1700

Data Index: 298996 (Open Web Text)

Max Activating Token Index: 958

Text #14

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3963. Min Act: -0.1700

Data Index: 3504085 (Open Web Text)

Max Activating Token Index: 1009

Text #15

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3066. Min Act: -0.1700

Data Index: 4325169 (Open Web Text)

Max Activating Token Index: 728

Text #16

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3435. Min Act: -0.1700

Data Index: 8817294 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3235. Min Act: -0.1700

Data Index: 99005 (Open Web Text)

Max Activating Token Index: 142

Text #18

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3337. Min Act: -0.1700

Data Index: 5159061 (Open Web Text)

Max Activating Token Index: 806

Text #19

Max Range: 4.6613. Min Range: -4.6613

Max Act: 4.3318. Min Act: -0.1700

Data Index: 1534867 (Open Web Text)

Max Activating Token Index: 194