Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2672 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.7639. Min Act: -0.1700

Data Index: 6952659 (Open Web Text)

Max Activating Token Index: 809

Text #1

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.5036. Min Act: -0.1700

Data Index: 1499170 (Open Web Text)

Max Activating Token Index: 963

Text #2

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.3553. Min Act: -0.1700

Data Index: 3092622 (Open Web Text)

Max Activating Token Index: 428

Text #3

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.3481. Min Act: -0.1700

Data Index: 6641362 (Open Web Text)

Max Activating Token Index: 205

Text #4

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.2722. Min Act: -0.1700

Data Index: 1690523 (Open Web Text)

Max Activating Token Index: 910

Text #5

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.1295. Min Act: -0.1700

Data Index: 2753964 (Open Web Text)

Max Activating Token Index: 304

Text #6

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.1113. Min Act: -0.1700

Data Index: 3364948 (Open Web Text)

Max Activating Token Index: 528

Text #7

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0901. Min Act: -0.1700

Data Index: 8628548 (Open Web Text)

Max Activating Token Index: 766

Text #8

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0813. Min Act: -0.1700

Data Index: 5345296 (Open Web Text)

Max Activating Token Index: 771

Text #9

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.1158. Min Act: -0.1700

Data Index: 3714667 (Open Web Text)

Max Activating Token Index: 441

Text #10

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.1016. Min Act: -0.1700

Data Index: 502905 (Open Web Text)

Max Activating Token Index: 364

Text #11

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0784. Min Act: -0.1700

Data Index: 6380939 (Open Web Text)

Max Activating Token Index: 867

Text #12

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0914. Min Act: -0.1700

Data Index: 6355370 (Open Web Text)

Max Activating Token Index: 395

Text #13

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0341. Min Act: -0.1700

Data Index: 4048059 (Open Web Text)

Max Activating Token Index: 483

Text #14

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0274. Min Act: -0.1700

Data Index: 1257911 (Open Web Text)

Max Activating Token Index: 878

Text #15

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0502. Min Act: -0.1700

Data Index: 2712758 (Open Web Text)

Max Activating Token Index: 502

Text #16

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0638. Min Act: -0.1700

Data Index: 7450595 (Open Web Text)

Max Activating Token Index: 844

Text #17

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0796. Min Act: -0.1700

Data Index: 615048 (Open Web Text)

Max Activating Token Index: 850

Text #18

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0457. Min Act: -0.1700

Data Index: 3385248 (Open Web Text)

Max Activating Token Index: 227

Text #19

Max Range: 5.7639. Min Range: -5.7639

Max Act: 5.0235. Min Act: -0.1700

Data Index: 5988628 (Open Web Text)

Max Activating Token Index: 419