Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 904 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.2111. Min Act: -0.1700

Data Index: 7900090 (Open Web Text)

Max Activating Token Index: 606

Text #1

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.1157. Min Act: -0.1700

Data Index: 6192423 (Open Web Text)

Max Activating Token Index: 472

Text #2

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.0883. Min Act: -0.1700

Data Index: 8427206 (Open Web Text)

Max Activating Token Index: 135

Text #3

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.1005. Min Act: -0.1700

Data Index: 7466053 (Open Web Text)

Max Activating Token Index: 202

Text #4

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.0485. Min Act: -0.1700

Data Index: 7301961 (Open Web Text)

Max Activating Token Index: 194

Text #5

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.0423. Min Act: -0.1700

Data Index: 7211380 (Open Web Text)

Max Activating Token Index: 123

Text #6

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.0159. Min Act: -0.1700

Data Index: 10773 (Open Web Text)

Max Activating Token Index: 349

Text #7

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9661. Min Act: -0.1700

Data Index: 8334100 (Open Web Text)

Max Activating Token Index: 30

Text #8

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9676. Min Act: -0.1700

Data Index: 2175891 (Open Web Text)

Max Activating Token Index: 796

Text #9

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9766. Min Act: -0.1700

Data Index: 5028781 (Open Web Text)

Max Activating Token Index: 200

Text #10

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9985. Min Act: -0.1700

Data Index: 5421958 (Open Web Text)

Max Activating Token Index: 461

Text #11

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9639. Min Act: -0.1700

Data Index: 4205430 (Open Web Text)

Max Activating Token Index: 557

Text #12

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9566. Min Act: -0.1700

Data Index: 6519140 (Open Web Text)

Max Activating Token Index: 201

Text #13

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9511. Min Act: -0.1700

Data Index: 5817408 (Open Web Text)

Max Activating Token Index: 475

Text #14

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9574. Min Act: -0.1700

Data Index: 712751 (Open Web Text)

Max Activating Token Index: 143

Text #15

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9686. Min Act: -0.1700

Data Index: 666035 (Open Web Text)

Max Activating Token Index: 541

Text #16

Max Range: 3.2111. Min Range: -3.2111

Max Act: 3.0059. Min Act: -0.1700

Data Index: 6811065 (Open Web Text)

Max Activating Token Index: 661

Text #17

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9534. Min Act: -0.1700

Data Index: 4990988 (Open Web Text)

Max Activating Token Index: 559

Text #18

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9140. Min Act: -0.1700

Data Index: 7792905 (Open Web Text)

Max Activating Token Index: 946

Text #19

Max Range: 3.2111. Min Range: -3.2111

Max Act: 2.9527. Min Act: -0.1700

Data Index: 6020643 (Open Web Text)

Max Activating Token Index: 55