Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1774 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.6222. Min Act: -0.1700

Data Index: 2029108 (Open Web Text)

Max Activating Token Index: 129

Text #1

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.5556. Min Act: -0.1700

Data Index: 8205737 (Open Web Text)

Max Activating Token Index: 194

Text #2

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.5315. Min Act: -0.1700

Data Index: 3547068 (Open Web Text)

Max Activating Token Index: 194

Text #3

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.5205. Min Act: -0.1700

Data Index: 5716539 (Open Web Text)

Max Activating Token Index: 133

Text #4

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.5429. Min Act: -0.1700

Data Index: 6769167 (Open Web Text)

Max Activating Token Index: 813

Text #5

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.5176. Min Act: -0.1700

Data Index: 5644813 (Open Web Text)

Max Activating Token Index: 307

Text #6

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4862. Min Act: -0.1700

Data Index: 415628 (Open Web Text)

Max Activating Token Index: 523

Text #7

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4587. Min Act: -0.1700

Data Index: 5337744 (Open Web Text)

Max Activating Token Index: 484

Text #8

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4828. Min Act: -0.1700

Data Index: 6857330 (Open Web Text)

Max Activating Token Index: 480

Text #9

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4978. Min Act: -0.1700

Data Index: 1173427 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4609. Min Act: -0.1700

Data Index: 1109801 (Open Web Text)

Max Activating Token Index: 688

Text #11

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4749. Min Act: -0.1700

Data Index: 3303955 (Open Web Text)

Max Activating Token Index: 140

Text #12

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4662. Min Act: -0.1700

Data Index: 2764600 (Open Web Text)

Max Activating Token Index: 883

Text #13

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4593. Min Act: -0.1700

Data Index: 3855509 (Open Web Text)

Max Activating Token Index: 287

Text #14

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4582. Min Act: -0.1700

Data Index: 2970274 (Open Web Text)

Max Activating Token Index: 383

Text #15

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4363. Min Act: -0.1700

Data Index: 940854 (Open Web Text)

Max Activating Token Index: 883

Text #16

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4349. Min Act: -0.1700

Data Index: 4800465 (Open Web Text)

Max Activating Token Index: 450

Text #17

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4419. Min Act: -0.1700

Data Index: 1977525 (Open Web Text)

Max Activating Token Index: 355

Text #18

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4227. Min Act: -0.1700

Data Index: 4740980 (Open Web Text)

Max Activating Token Index: 211

Text #19

Max Range: 1.6222. Min Range: -1.6222

Max Act: 1.4560. Min Act: -0.1700

Data Index: 3303436 (Open Web Text)

Max Activating Token Index: 704