Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2476 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.5460. Min Act: -0.1700

Data Index: 7718983 (Open Web Text)

Max Activating Token Index: 442

Text #1

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.4902. Min Act: -0.1700

Data Index: 6855448 (Open Web Text)

Max Activating Token Index: 495

Text #2

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.4553. Min Act: -0.1700

Data Index: 4278408 (Open Web Text)

Max Activating Token Index: 307

Text #3

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.3898. Min Act: -0.1700

Data Index: 8054812 (Open Web Text)

Max Activating Token Index: 144

Text #4

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.3702. Min Act: -0.1700

Data Index: 4937133 (Open Web Text)

Max Activating Token Index: 897

Text #5

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.3912. Min Act: -0.1700

Data Index: 7382461 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2851. Min Act: -0.1700

Data Index: 8349918 (Open Web Text)

Max Activating Token Index: 692

Text #7

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2849. Min Act: -0.1700

Data Index: 3027654 (Open Web Text)

Max Activating Token Index: 754

Text #8

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2830. Min Act: -0.1700

Data Index: 7947804 (Open Web Text)

Max Activating Token Index: 788

Text #9

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2307. Min Act: -0.1700

Data Index: 842780 (Open Web Text)

Max Activating Token Index: 847

Text #10

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2477. Min Act: -0.1700

Data Index: 4223061 (Open Web Text)

Max Activating Token Index: 82

Text #11

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2466. Min Act: -0.1700

Data Index: 7922351 (Open Web Text)

Max Activating Token Index: 638

Text #12

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2705. Min Act: -0.1700

Data Index: 437633 (Open Web Text)

Max Activating Token Index: 75

Text #13

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2521. Min Act: -0.1700

Data Index: 126662 (Open Web Text)

Max Activating Token Index: 970

Text #14

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.1999. Min Act: -0.1700

Data Index: 6469288 (Open Web Text)

Max Activating Token Index: 387

Text #15

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2638. Min Act: -0.1700

Data Index: 4102686 (Open Web Text)

Max Activating Token Index: 371

Text #16

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2085. Min Act: -0.1700

Data Index: 178712 (Open Web Text)

Max Activating Token Index: 299

Text #17

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.2680. Min Act: -0.1700

Data Index: 2701704 (Open Web Text)

Max Activating Token Index: 479

Text #18

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.1951. Min Act: -0.1700

Data Index: 917917 (Open Web Text)

Max Activating Token Index: 305

Text #19

Max Range: 6.5460. Min Range: -6.5460

Max Act: 6.1798. Min Act: -0.1700

Data Index: 4341035 (Open Web Text)

Max Activating Token Index: 612