Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2434 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2784. Min Act: -0.1700

Data Index: 6029453 (Open Web Text)

Max Activating Token Index: 672

Text #1

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2385. Min Act: -0.1700

Data Index: 5287133 (Open Web Text)

Max Activating Token Index: 858

Text #2

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2237. Min Act: -0.1700

Data Index: 1352275 (Open Web Text)

Max Activating Token Index: 319

Text #3

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2252. Min Act: -0.1700

Data Index: 8774957 (Open Web Text)

Max Activating Token Index: 396

Text #4

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2212. Min Act: -0.1700

Data Index: 4996389 (Open Web Text)

Max Activating Token Index: 917

Text #5

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.2042. Min Act: -0.1700

Data Index: 3337968 (Open Web Text)

Max Activating Token Index: 585

Text #6

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1952. Min Act: -0.1700

Data Index: 4812964 (Open Web Text)

Max Activating Token Index: 663

Text #7

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1971. Min Act: -0.1700

Data Index: 782672 (Open Web Text)

Max Activating Token Index: 600

Text #8

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1550. Min Act: -0.1700

Data Index: 3008349 (Open Web Text)

Max Activating Token Index: 636

Text #9

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1753. Min Act: -0.1700

Data Index: 718705 (Open Web Text)

Max Activating Token Index: 974

Text #10

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1531. Min Act: -0.1700

Data Index: 3180019 (Open Web Text)

Max Activating Token Index: 977

Text #11

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1895. Min Act: -0.1700

Data Index: 5509942 (Open Web Text)

Max Activating Token Index: 466

Text #12

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1701. Min Act: -0.1700

Data Index: 8458487 (Open Web Text)

Max Activating Token Index: 527

Text #13

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1629. Min Act: -0.1700

Data Index: 2863304 (Open Web Text)

Max Activating Token Index: 451

Text #14

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1650. Min Act: -0.1700

Data Index: 2676775 (Open Web Text)

Max Activating Token Index: 707

Text #15

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1352. Min Act: -0.1700

Data Index: 7629045 (Open Web Text)

Max Activating Token Index: 893

Text #16

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1447. Min Act: -0.1700

Data Index: 4127566 (Open Web Text)

Max Activating Token Index: 1008

Text #17

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1389. Min Act: -0.1700

Data Index: 2411709 (Open Web Text)

Max Activating Token Index: 942

Text #18

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1291. Min Act: -0.1700

Data Index: 689627 (Open Web Text)

Max Activating Token Index: 248

Text #19

Max Range: 3.2784. Min Range: -3.2784

Max Act: 3.1358. Min Act: -0.1700

Data Index: 6906433 (Open Web Text)

Max Activating Token Index: 960