Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 105 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.5272. Min Act: -0.1700

Data Index: 7316938 (Open Web Text)

Max Activating Token Index: 863

Text #1

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.6425. Min Act: -0.1700

Data Index: 103493 (Open Web Text)

Max Activating Token Index: 858

Text #2

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4477. Min Act: -0.1700

Data Index: 6879246 (Open Web Text)

Max Activating Token Index: 816

Text #3

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4847. Min Act: -0.1700

Data Index: 3320207 (Open Web Text)

Max Activating Token Index: 503

Text #4

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.5118. Min Act: -0.1700

Data Index: 831168 (Open Web Text)

Max Activating Token Index: 375

Text #5

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4385. Min Act: -0.1700

Data Index: 7746469 (Open Web Text)

Max Activating Token Index: 569

Text #6

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4532. Min Act: -0.1700

Data Index: 1392283 (Open Web Text)

Max Activating Token Index: 26

Text #7

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4836. Min Act: -0.1700

Data Index: 4506221 (Open Web Text)

Max Activating Token Index: 1015

Text #8

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4413. Min Act: -0.1700

Data Index: 1062976 (Open Web Text)

Max Activating Token Index: 765

Text #9

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4843. Min Act: -0.1700

Data Index: 7005840 (Open Web Text)

Max Activating Token Index: 806

Text #10

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4702. Min Act: -0.1700

Data Index: 1972678 (Open Web Text)

Max Activating Token Index: 308

Text #11

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4298. Min Act: -0.1700

Data Index: 5387799 (Open Web Text)

Max Activating Token Index: 465

Text #12

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4284. Min Act: -0.1700

Data Index: 4652263 (Open Web Text)

Max Activating Token Index: 340

Text #13

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4292. Min Act: -0.1700

Data Index: 6660883 (Open Web Text)

Max Activating Token Index: 707

Text #14

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4091. Min Act: -0.1700

Data Index: 4224894 (Open Web Text)

Max Activating Token Index: 97

Text #15

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4032. Min Act: -0.1700

Data Index: 7591273 (Open Web Text)

Max Activating Token Index: 21

Text #16

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.3760. Min Act: -0.1700

Data Index: 2587635 (Open Web Text)

Max Activating Token Index: 216

Text #17

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.3849. Min Act: -0.1700

Data Index: 5008133 (Open Web Text)

Max Activating Token Index: 30

Text #18

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.4051. Min Act: -0.1700

Data Index: 6471358 (Open Web Text)

Max Activating Token Index: 1011

Text #19

Max Range: 4.6425. Min Range: -4.6425

Max Act: 4.3734. Min Act: -0.1700

Data Index: 5886767 (Open Web Text)

Max Activating Token Index: 643