Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1824 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3763. Min Act: -0.1700

Data Index: 384877 (Open Web Text)

Max Activating Token Index: 154

Text #1

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3698. Min Act: -0.1700

Data Index: 6718453 (Open Web Text)

Max Activating Token Index: 656

Text #2

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3324. Min Act: -0.1700

Data Index: 5041329 (Open Web Text)

Max Activating Token Index: 771

Text #3

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3114. Min Act: -0.1700

Data Index: 7599225 (Open Web Text)

Max Activating Token Index: 329

Text #4

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3085. Min Act: -0.1700

Data Index: 749546 (Open Web Text)

Max Activating Token Index: 336

Text #5

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3260. Min Act: -0.1700

Data Index: 1126033 (Open Web Text)

Max Activating Token Index: 556

Text #6

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.3169. Min Act: -0.1700

Data Index: 6495261 (Open Web Text)

Max Activating Token Index: 271

Text #7

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2903. Min Act: -0.1700

Data Index: 77952 (Open Web Text)

Max Activating Token Index: 502

Text #8

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2965. Min Act: -0.1700

Data Index: 4298475 (Open Web Text)

Max Activating Token Index: 906

Text #9

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2879. Min Act: -0.1700

Data Index: 4548269 (Open Web Text)

Max Activating Token Index: 631

Text #10

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2642. Min Act: -0.1700

Data Index: 1418746 (Open Web Text)

Max Activating Token Index: 793

Text #11

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2560. Min Act: -0.1700

Data Index: 2455799 (Open Web Text)

Max Activating Token Index: 477

Text #12

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2505. Min Act: -0.1700

Data Index: 7302527 (Open Web Text)

Max Activating Token Index: 669

Text #13

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2323. Min Act: -0.1700

Data Index: 7012432 (Open Web Text)

Max Activating Token Index: 573

Text #14

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2320. Min Act: -0.1700

Data Index: 3352574 (Open Web Text)

Max Activating Token Index: 592

Text #15

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2430. Min Act: -0.1700

Data Index: 8706968 (Open Web Text)

Max Activating Token Index: 587

Text #16

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2506. Min Act: -0.1700

Data Index: 4073127 (Open Web Text)

Max Activating Token Index: 16

Text #17

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2476. Min Act: -0.1700

Data Index: 1355390 (Open Web Text)

Max Activating Token Index: 921

Text #18

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2341. Min Act: -0.1700

Data Index: 2375313 (Open Web Text)

Max Activating Token Index: 574

Text #19

Max Range: 3.3763. Min Range: -3.3763

Max Act: 3.2279. Min Act: -0.1700

Data Index: 6785886 (Open Web Text)

Max Activating Token Index: 117