Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2018 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.9254. Min Act: -0.1700

Data Index: 726001 (Open Web Text)

Max Activating Token Index: 180

Text #1

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.8330. Min Act: -0.1700

Data Index: 6317834 (Open Web Text)

Max Activating Token Index: 471

Text #2

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.9040. Min Act: -0.1700

Data Index: 8706405 (Open Web Text)

Max Activating Token Index: 395

Text #3

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7551. Min Act: -0.1700

Data Index: 3555763 (Open Web Text)

Max Activating Token Index: 741

Text #4

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.8397. Min Act: -0.1700

Data Index: 680410 (Open Web Text)

Max Activating Token Index: 287

Text #5

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.8112. Min Act: -0.1700

Data Index: 4525234 (Open Web Text)

Max Activating Token Index: 115

Text #6

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.8008. Min Act: -0.1700

Data Index: 4143535 (Open Web Text)

Max Activating Token Index: 262

Text #7

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7837. Min Act: -0.1700

Data Index: 3811654 (Open Web Text)

Max Activating Token Index: 460

Text #8

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7669. Min Act: -0.1700

Data Index: 3236452 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7669. Min Act: -0.1700

Data Index: 3706498 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7950. Min Act: -0.1700

Data Index: 7269357 (Open Web Text)

Max Activating Token Index: 475

Text #11

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7627. Min Act: -0.1700

Data Index: 8755757 (Open Web Text)

Max Activating Token Index: 98

Text #12

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7640. Min Act: -0.1700

Data Index: 4854686 (Open Web Text)

Max Activating Token Index: 715

Text #13

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7605. Min Act: -0.1700

Data Index: 8601202 (Open Web Text)

Max Activating Token Index: 531

Text #14

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7683. Min Act: -0.1700

Data Index: 7010811 (Open Web Text)

Max Activating Token Index: 527

Text #15

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7350. Min Act: -0.1700

Data Index: 6207089 (Open Web Text)

Max Activating Token Index: 22

Text #16

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7446. Min Act: -0.1700

Data Index: 3149189 (Open Web Text)

Max Activating Token Index: 538

Text #17

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7256. Min Act: -0.1700

Data Index: 5022801 (Open Web Text)

Max Activating Token Index: 836

Text #18

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7066. Min Act: -0.1700

Data Index: 7845791 (Open Web Text)

Max Activating Token Index: 414

Text #19

Max Range: 3.9254. Min Range: -3.9254

Max Act: 3.7379. Min Act: -0.1700

Data Index: 4334125 (Open Web Text)

Max Activating Token Index: 420