Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 332 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3855. Min Act: -0.1700

Data Index: 3335275 (Open Web Text)

Max Activating Token Index: 332

Text #1

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3310. Min Act: -0.1700

Data Index: 609158 (Open Web Text)

Max Activating Token Index: 532

Text #2

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3621. Min Act: -0.1700

Data Index: 4182319 (Open Web Text)

Max Activating Token Index: 419

Text #3

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3466. Min Act: -0.1700

Data Index: 2528184 (Open Web Text)

Max Activating Token Index: 885

Text #4

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3508. Min Act: -0.1700

Data Index: 3747070 (Open Web Text)

Max Activating Token Index: 691

Text #5

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3434. Min Act: -0.1700

Data Index: 1711156 (Open Web Text)

Max Activating Token Index: 453

Text #6

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3088. Min Act: -0.1700

Data Index: 5685565 (Open Web Text)

Max Activating Token Index: 713

Text #7

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.3328. Min Act: -0.1700

Data Index: 3486896 (Open Web Text)

Max Activating Token Index: 603

Text #8

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2967. Min Act: -0.1700

Data Index: 6838702 (Open Web Text)

Max Activating Token Index: 255

Text #9

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2495. Min Act: -0.1700

Data Index: 8481447 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2712. Min Act: -0.1700

Data Index: 1994364 (Open Web Text)

Max Activating Token Index: 938

Text #11

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2452. Min Act: -0.1700

Data Index: 5401493 (Open Web Text)

Max Activating Token Index: 971

Text #12

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2651. Min Act: -0.1700

Data Index: 5105124 (Open Web Text)

Max Activating Token Index: 41

Text #13

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2444. Min Act: -0.1700

Data Index: 3349851 (Open Web Text)

Max Activating Token Index: 668

Text #14

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2486. Min Act: -0.1700

Data Index: 8581474 (Open Web Text)

Max Activating Token Index: 776

Text #15

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2583. Min Act: -0.1700

Data Index: 8162103 (Open Web Text)

Max Activating Token Index: 447

Text #16

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2453. Min Act: -0.1700

Data Index: 7144678 (Open Web Text)

Max Activating Token Index: 552

Text #17

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2497. Min Act: -0.1700

Data Index: 1924665 (Open Web Text)

Max Activating Token Index: 643

Text #18

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2282. Min Act: -0.1700

Data Index: 2030322 (Open Web Text)

Max Activating Token Index: 430

Text #19

Max Range: 3.3855. Min Range: -3.3855

Max Act: 3.2446. Min Act: -0.1700

Data Index: 7084748 (Open Web Text)

Max Activating Token Index: 963