Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 810 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8668. Min Act: -0.1700

Data Index: 3608619 (Open Web Text)

Max Activating Token Index: 538

Text #1

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8756. Min Act: -0.1700

Data Index: 7454188 (Open Web Text)

Max Activating Token Index: 619

Text #2

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8261. Min Act: -0.1700

Data Index: 2334994 (Open Web Text)

Max Activating Token Index: 683

Text #3

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8326. Min Act: -0.1700

Data Index: 7311132 (Open Web Text)

Max Activating Token Index: 498

Text #4

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8166. Min Act: -0.1700

Data Index: 383447 (Open Web Text)

Max Activating Token Index: 264

Text #5

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.8082. Min Act: -0.1700

Data Index: 447847 (Open Web Text)

Max Activating Token Index: 41

Text #6

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7543. Min Act: -0.1700

Data Index: 3307967 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7529. Min Act: -0.1700

Data Index: 1000653 (Open Web Text)

Max Activating Token Index: 244

Text #8

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7462. Min Act: -0.1700

Data Index: 989312 (Open Web Text)

Max Activating Token Index: 171

Text #9

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7509. Min Act: -0.1700

Data Index: 4155870 (Open Web Text)

Max Activating Token Index: 708

Text #10

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7695. Min Act: -0.1700

Data Index: 3841516 (Open Web Text)

Max Activating Token Index: 876

Text #11

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7470. Min Act: -0.1700

Data Index: 7177179 (Open Web Text)

Max Activating Token Index: 695

Text #12

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7374. Min Act: -0.1700

Data Index: 296059 (Open Web Text)

Max Activating Token Index: 587

Text #13

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7270. Min Act: -0.1700

Data Index: 8060856 (Open Web Text)

Max Activating Token Index: 267

Text #14

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7207. Min Act: -0.1700

Data Index: 229466 (Open Web Text)

Max Activating Token Index: 709

Text #15

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7345. Min Act: -0.1700

Data Index: 4378841 (Open Web Text)

Max Activating Token Index: 129

Text #16

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.6854. Min Act: -0.1700

Data Index: 352051 (Open Web Text)

Max Activating Token Index: 635

Text #17

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.6998. Min Act: -0.1700

Data Index: 6670082 (Open Web Text)

Max Activating Token Index: 298

Text #18

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.6838. Min Act: -0.1700

Data Index: 4114267 (Open Web Text)

Max Activating Token Index: 561

Text #19

Max Range: 4.8756. Min Range: -4.8756

Max Act: 4.7033. Min Act: -0.1700

Data Index: 6355092 (Open Web Text)

Max Activating Token Index: 571