Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1218 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.3639. Min Act: -0.1700

Data Index: 6150830 (Open Web Text)

Max Activating Token Index: 295

Text #1

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.3067. Min Act: -0.1700

Data Index: 3787541 (Open Web Text)

Max Activating Token Index: 378

Text #2

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.2869. Min Act: -0.1700

Data Index: 2898844 (Open Web Text)

Max Activating Token Index: 420

Text #3

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.2202. Min Act: -0.1700

Data Index: 7240208 (Open Web Text)

Max Activating Token Index: 644

Text #4

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1913. Min Act: -0.1700

Data Index: 3614203 (Open Web Text)

Max Activating Token Index: 851

Text #5

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1747. Min Act: -0.1700

Data Index: 2824081 (Open Web Text)

Max Activating Token Index: 727

Text #6

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1873. Min Act: -0.1700

Data Index: 2513240 (Open Web Text)

Max Activating Token Index: 533

Text #7

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1817. Min Act: -0.1700

Data Index: 4432907 (Open Web Text)

Max Activating Token Index: 212

Text #8

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1966. Min Act: -0.1700

Data Index: 1212907 (Open Web Text)

Max Activating Token Index: 488

Text #9

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1500. Min Act: -0.1700

Data Index: 5500495 (Open Web Text)

Max Activating Token Index: 782

Text #10

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1423. Min Act: -0.1700

Data Index: 2650227 (Open Web Text)

Max Activating Token Index: 484

Text #11

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1619. Min Act: -0.1700

Data Index: 8809278 (Open Web Text)

Max Activating Token Index: 550

Text #12

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1596. Min Act: -0.1700

Data Index: 4290234 (Open Web Text)

Max Activating Token Index: 172

Text #13

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1566. Min Act: -0.1700

Data Index: 8727399 (Open Web Text)

Max Activating Token Index: 415

Text #14

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1039. Min Act: -0.1700

Data Index: 3856663 (Open Web Text)

Max Activating Token Index: 609

Text #15

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1261. Min Act: -0.1700

Data Index: 417970 (Open Web Text)

Max Activating Token Index: 929

Text #16

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1140. Min Act: -0.1700

Data Index: 6689455 (Open Web Text)

Max Activating Token Index: 505

Text #17

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1120. Min Act: -0.1700

Data Index: 6442124 (Open Web Text)

Max Activating Token Index: 551

Text #18

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1043. Min Act: -0.1700

Data Index: 7535016 (Open Web Text)

Max Activating Token Index: 447

Text #19

Max Range: 3.3639. Min Range: -3.3639

Max Act: 3.1548. Min Act: -0.1700

Data Index: 8140723 (Open Web Text)

Max Activating Token Index: 827