Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1544 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.6052. Min Act: -0.1700

Data Index: 4234972 (Open Web Text)

Max Activating Token Index: 987

Text #1

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.5294. Min Act: -0.1700

Data Index: 4057180 (Open Web Text)

Max Activating Token Index: 932

Text #2

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.4846. Min Act: -0.1700

Data Index: 8765951 (Open Web Text)

Max Activating Token Index: 780

Text #3

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.4847. Min Act: -0.1700

Data Index: 8213638 (Open Web Text)

Max Activating Token Index: 575

Text #4

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.4229. Min Act: -0.1700

Data Index: 7245262 (Open Web Text)

Max Activating Token Index: 484

Text #5

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3995. Min Act: -0.1700

Data Index: 6898546 (Open Web Text)

Max Activating Token Index: 639

Text #6

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3885. Min Act: -0.1700

Data Index: 7765276 (Open Web Text)

Max Activating Token Index: 401

Text #7

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3972. Min Act: -0.1700

Data Index: 1465428 (Open Web Text)

Max Activating Token Index: 578

Text #8

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.4001. Min Act: -0.1700

Data Index: 7264101 (Open Web Text)

Max Activating Token Index: 264

Text #9

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3927. Min Act: -0.1700

Data Index: 6258477 (Open Web Text)

Max Activating Token Index: 618

Text #10

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3968. Min Act: -0.1700

Data Index: 3724451 (Open Web Text)

Max Activating Token Index: 313

Text #11

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3734. Min Act: -0.1700

Data Index: 4035337 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3868. Min Act: -0.1700

Data Index: 7006124 (Open Web Text)

Max Activating Token Index: 676

Text #13

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3698. Min Act: -0.1700

Data Index: 1532704 (Open Web Text)

Max Activating Token Index: 346

Text #14

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3524. Min Act: -0.1700

Data Index: 5039086 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3507. Min Act: -0.1700

Data Index: 7240362 (Open Web Text)

Max Activating Token Index: 700

Text #16

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3488. Min Act: -0.1700

Data Index: 7383524 (Open Web Text)

Max Activating Token Index: 38

Text #17

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3504. Min Act: -0.1700

Data Index: 3294547 (Open Web Text)

Max Activating Token Index: 301

Text #18

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3535. Min Act: -0.1700

Data Index: 7584123 (Open Web Text)

Max Activating Token Index: 305

Text #19

Max Range: 3.6052. Min Range: -3.6052

Max Act: 3.3679. Min Act: -0.1700

Data Index: 7858345 (Open Web Text)

Max Activating Token Index: 442