Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2128 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.5475. Min Act: -0.1700

Data Index: 7312789 (Open Web Text)

Max Activating Token Index: 570

Text #1

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.3946. Min Act: -0.1700

Data Index: 8589563 (Open Web Text)

Max Activating Token Index: 911

Text #2

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.3122. Min Act: -0.1700

Data Index: 8639325 (Open Web Text)

Max Activating Token Index: 387

Text #3

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2317. Min Act: -0.1700

Data Index: 390148 (Open Web Text)

Max Activating Token Index: 500

Text #4

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2708. Min Act: -0.1700

Data Index: 595001 (Open Web Text)

Max Activating Token Index: 755

Text #5

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2467. Min Act: -0.1700

Data Index: 7574380 (Open Web Text)

Max Activating Token Index: 522

Text #6

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2339. Min Act: -0.1700

Data Index: 6668363 (Open Web Text)

Max Activating Token Index: 906

Text #7

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2611. Min Act: -0.1700

Data Index: 2703561 (Open Web Text)

Max Activating Token Index: 362

Text #8

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2059. Min Act: -0.1700

Data Index: 131455 (Open Web Text)

Max Activating Token Index: 780

Text #9

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1865. Min Act: -0.1700

Data Index: 2595077 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2021. Min Act: -0.1700

Data Index: 5756763 (Open Web Text)

Max Activating Token Index: 100

Text #11

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2452. Min Act: -0.1700

Data Index: 3893285 (Open Web Text)

Max Activating Token Index: 993

Text #12

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2010. Min Act: -0.1700

Data Index: 8574845 (Open Web Text)

Max Activating Token Index: 755

Text #13

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1916. Min Act: -0.1700

Data Index: 3644292 (Open Web Text)

Max Activating Token Index: 393

Text #14

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2036. Min Act: -0.1700

Data Index: 6690138 (Open Web Text)

Max Activating Token Index: 893

Text #15

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.2129. Min Act: -0.1700

Data Index: 1690891 (Open Web Text)

Max Activating Token Index: 404

Text #16

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1565. Min Act: -0.1700

Data Index: 2816652 (Open Web Text)

Max Activating Token Index: 791

Text #17

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1488. Min Act: -0.1700

Data Index: 5808685 (Open Web Text)

Max Activating Token Index: 246

Text #18

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1678. Min Act: -0.1700

Data Index: 2048995 (Open Web Text)

Max Activating Token Index: 509

Text #19

Max Range: 4.5475. Min Range: -4.5475

Max Act: 4.1713. Min Act: -0.1700

Data Index: 6005608 (Open Web Text)

Max Activating Token Index: 54