Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 202 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1398. Min Range: -3.1398

Max Act: 3.1398. Min Act: -0.1700

Data Index: 1676368 (Open Web Text)

Max Activating Token Index: 65

Text #1

Max Range: 3.1398. Min Range: -3.1398

Max Act: 3.0685. Min Act: -0.1700

Data Index: 672901 (Open Web Text)

Max Activating Token Index: 23

Text #2

Max Range: 3.1398. Min Range: -3.1398

Max Act: 3.0450. Min Act: -0.1700

Data Index: 3098386 (Open Web Text)

Max Activating Token Index: 356

Text #3

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9978. Min Act: -0.1700

Data Index: 6806384 (Open Web Text)

Max Activating Token Index: 17

Text #4

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9788. Min Act: -0.1700

Data Index: 1772460 (Open Web Text)

Max Activating Token Index: 57

Text #5

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9734. Min Act: -0.1700

Data Index: 5903849 (Open Web Text)

Max Activating Token Index: 18

Text #6

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9531. Min Act: -0.1700

Data Index: 5759063 (Open Web Text)

Max Activating Token Index: 620

Text #7

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9334. Min Act: -0.1700

Data Index: 3876928 (Open Web Text)

Max Activating Token Index: 22

Text #8

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9347. Min Act: -0.1700

Data Index: 2964028 (Open Web Text)

Max Activating Token Index: 498

Text #9

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.9072. Min Act: -0.1700

Data Index: 3398675 (Open Web Text)

Max Activating Token Index: 889

Text #10

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8803. Min Act: -0.1700

Data Index: 8411431 (Open Web Text)

Max Activating Token Index: 35

Text #11

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8809. Min Act: -0.1700

Data Index: 5570498 (Open Web Text)

Max Activating Token Index: 178

Text #12

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8824. Min Act: -0.1700

Data Index: 6873986 (Open Web Text)

Max Activating Token Index: 422

Text #13

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8950. Min Act: -0.1700

Data Index: 7118837 (Open Web Text)

Max Activating Token Index: 655

Text #14

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8544. Min Act: -0.1700

Data Index: 2341638 (Open Web Text)

Max Activating Token Index: 472

Text #15

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8675. Min Act: -0.1700

Data Index: 5559841 (Open Web Text)

Max Activating Token Index: 23

Text #16

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8528. Min Act: -0.1700

Data Index: 2569859 (Open Web Text)

Max Activating Token Index: 792

Text #17

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8565. Min Act: -0.1700

Data Index: 6414268 (Open Web Text)

Max Activating Token Index: 507

Text #18

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8649. Min Act: -0.1700

Data Index: 4551850 (Open Web Text)

Max Activating Token Index: 195

Text #19

Max Range: 3.1398. Min Range: -3.1398

Max Act: 2.8503. Min Act: -0.1700

Data Index: 1090908 (Open Web Text)

Max Activating Token Index: 787