Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 855 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.8216. Min Act: -0.1700

Data Index: 1487591 (Open Web Text)

Max Activating Token Index: 827

Text #1

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4157. Min Act: -0.1700

Data Index: 1793655 (Open Web Text)

Max Activating Token Index: 469

Text #2

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4459. Min Act: -0.1700

Data Index: 1827638 (Open Web Text)

Max Activating Token Index: 255

Text #3

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4336. Min Act: -0.1700

Data Index: 3694579 (Open Web Text)

Max Activating Token Index: 31

Text #4

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4378. Min Act: -0.1700

Data Index: 310503 (Open Web Text)

Max Activating Token Index: 280

Text #5

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4308. Min Act: -0.1700

Data Index: 4641409 (Open Web Text)

Max Activating Token Index: 798

Text #6

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4312. Min Act: -0.1700

Data Index: 4361008 (Open Web Text)

Max Activating Token Index: 750

Text #7

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3954. Min Act: -0.1700

Data Index: 996188 (Open Web Text)

Max Activating Token Index: 101

Text #8

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4124. Min Act: -0.1700

Data Index: 5787615 (Open Web Text)

Max Activating Token Index: 900

Text #9

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.4128. Min Act: -0.1700

Data Index: 4253474 (Open Web Text)

Max Activating Token Index: 682

Text #10

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3462. Min Act: -0.1700

Data Index: 7696703 (Open Web Text)

Max Activating Token Index: 852

Text #11

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3426. Min Act: -0.1700

Data Index: 2803801 (Open Web Text)

Max Activating Token Index: 854

Text #12

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3846. Min Act: -0.1700

Data Index: 8206799 (Open Web Text)

Max Activating Token Index: 1001

Text #13

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3631. Min Act: -0.1700

Data Index: 6368313 (Open Web Text)

Max Activating Token Index: 566

Text #14

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3428. Min Act: -0.1700

Data Index: 7319566 (Open Web Text)

Max Activating Token Index: 326

Text #15

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3486. Min Act: -0.1700

Data Index: 321159 (Open Web Text)

Max Activating Token Index: 482

Text #16

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3460. Min Act: -0.1700

Data Index: 3729050 (Open Web Text)

Max Activating Token Index: 735

Text #17

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3325. Min Act: -0.1700

Data Index: 6712343 (Open Web Text)

Max Activating Token Index: 846

Text #18

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3114. Min Act: -0.1700

Data Index: 3956321 (Open Web Text)

Max Activating Token Index: 440

Text #19

Max Range: 4.8216. Min Range: -4.8216

Max Act: 4.3266. Min Act: -0.1700

Data Index: 6179228 (Open Web Text)

Max Activating Token Index: 339