Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2105 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.8624. Min Act: -0.1700

Data Index: 6820228 (Open Web Text)

Max Activating Token Index: 531

Text #1

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.5575. Min Act: -0.1700

Data Index: 2555359 (Open Web Text)

Max Activating Token Index: 607

Text #2

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.5465. Min Act: -0.1700

Data Index: 573232 (Open Web Text)

Max Activating Token Index: 244

Text #3

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.3832. Min Act: -0.1700

Data Index: 609953 (Open Web Text)

Max Activating Token Index: 457

Text #4

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.4953. Min Act: -0.1700

Data Index: 6354657 (Open Web Text)

Max Activating Token Index: 463

Text #5

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.3919. Min Act: -0.1700

Data Index: 7458640 (Open Web Text)

Max Activating Token Index: 325

Text #6

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.3052. Min Act: -0.1700

Data Index: 6687230 (Open Web Text)

Max Activating Token Index: 808

Text #7

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.3165. Min Act: -0.1700

Data Index: 8806082 (Open Web Text)

Max Activating Token Index: 689

Text #8

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.2838. Min Act: -0.1700

Data Index: 4162147 (Open Web Text)

Max Activating Token Index: 235

Text #9

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.2537. Min Act: -0.1700

Data Index: 160341 (Open Web Text)

Max Activating Token Index: 926

Text #10

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.2007. Min Act: -0.1700

Data Index: 4183744 (Open Web Text)

Max Activating Token Index: 324

Text #11

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.2233. Min Act: -0.1700

Data Index: 1800177 (Open Web Text)

Max Activating Token Index: 80

Text #12

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0942. Min Act: -0.1700

Data Index: 337010 (Open Web Text)

Max Activating Token Index: 394

Text #13

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0469. Min Act: -0.1700

Data Index: 4344147 (Open Web Text)

Max Activating Token Index: 595

Text #14

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.1599. Min Act: -0.1700

Data Index: 2080572 (Open Web Text)

Max Activating Token Index: 203

Text #15

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0441. Min Act: -0.1700

Data Index: 2472605 (Open Web Text)

Max Activating Token Index: 583

Text #16

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0912. Min Act: -0.1700

Data Index: 473133 (Open Web Text)

Max Activating Token Index: 416

Text #17

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0568. Min Act: -0.1700

Data Index: 6341668 (Open Web Text)

Max Activating Token Index: 746

Text #18

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.1051. Min Act: -0.1700

Data Index: 2813485 (Open Web Text)

Max Activating Token Index: 773

Text #19

Max Range: 7.8624. Min Range: -7.8624

Max Act: 7.0522. Min Act: -0.1700

Data Index: 2750999 (Open Web Text)

Max Activating Token Index: 512