Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 743 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.3843. Min Act: -0.1700

Data Index: 6636067 (Open Web Text)

Max Activating Token Index: 255

Text #1

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.3802. Min Act: -0.1700

Data Index: 6091166 (Open Web Text)

Max Activating Token Index: 174

Text #2

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.3933. Min Act: -0.1700

Data Index: 6324122 (Open Web Text)

Max Activating Token Index: 708

Text #3

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.3362. Min Act: -0.1700

Data Index: 765001 (Open Web Text)

Max Activating Token Index: 556

Text #4

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.3457. Min Act: -0.1700

Data Index: 5819312 (Open Web Text)

Max Activating Token Index: 14

Text #5

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2894. Min Act: -0.1700

Data Index: 6418607 (Open Web Text)

Max Activating Token Index: 679

Text #6

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2938. Min Act: -0.1700

Data Index: 6505365 (Open Web Text)

Max Activating Token Index: 302

Text #7

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2821. Min Act: -0.1700

Data Index: 2055385 (Open Web Text)

Max Activating Token Index: 510

Text #8

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2629. Min Act: -0.1700

Data Index: 1166508 (Open Web Text)

Max Activating Token Index: 578

Text #9

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2410. Min Act: -0.1700

Data Index: 7127523 (Open Web Text)

Max Activating Token Index: 476

Text #10

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2544. Min Act: -0.1700

Data Index: 765003 (Open Web Text)

Max Activating Token Index: 876

Text #11

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2530. Min Act: -0.1700

Data Index: 5967610 (Open Web Text)

Max Activating Token Index: 563

Text #12

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2377. Min Act: -0.1700

Data Index: 2295564 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2217. Min Act: -0.1700

Data Index: 6324120 (Open Web Text)

Max Activating Token Index: 470

Text #14

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2448. Min Act: -0.1700

Data Index: 2745557 (Open Web Text)

Max Activating Token Index: 411

Text #15

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2267. Min Act: -0.1700

Data Index: 1937703 (Open Web Text)

Max Activating Token Index: 477

Text #16

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2176. Min Act: -0.1700

Data Index: 207784 (Open Web Text)

Max Activating Token Index: 258

Text #17

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2433. Min Act: -0.1700

Data Index: 1224821 (Open Web Text)

Max Activating Token Index: 755

Text #18

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2130. Min Act: -0.1700

Data Index: 4464639 (Open Web Text)

Max Activating Token Index: 777

Text #19

Max Range: 3.3933. Min Range: -3.3933

Max Act: 3.2001. Min Act: -0.1700

Data Index: 2391428 (Open Web Text)

Max Activating Token Index: 270