Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1182 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.8991. Min Act: -0.1700

Data Index: 6588808 (Open Web Text)

Max Activating Token Index: 605

Text #1

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.8881. Min Act: -0.1700

Data Index: 6630179 (Open Web Text)

Max Activating Token Index: 989

Text #2

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7063. Min Act: -0.1700

Data Index: 7512613 (Open Web Text)

Max Activating Token Index: 906

Text #3

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7812. Min Act: -0.1700

Data Index: 7724183 (Open Web Text)

Max Activating Token Index: 951

Text #4

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.8328. Min Act: -0.1700

Data Index: 5403866 (Open Web Text)

Max Activating Token Index: 864

Text #5

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7683. Min Act: -0.1700

Data Index: 7788772 (Open Web Text)

Max Activating Token Index: 933

Text #6

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7944. Min Act: -0.1700

Data Index: 2688779 (Open Web Text)

Max Activating Token Index: 263

Text #7

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7755. Min Act: -0.1700

Data Index: 3815155 (Open Web Text)

Max Activating Token Index: 612

Text #8

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7415. Min Act: -0.1700

Data Index: 1096588 (Open Web Text)

Max Activating Token Index: 101

Text #9

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.6769. Min Act: -0.1700

Data Index: 6917693 (Open Web Text)

Max Activating Token Index: 611

Text #10

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.8027. Min Act: -0.1700

Data Index: 4120575 (Open Web Text)

Max Activating Token Index: 893

Text #11

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.6927. Min Act: -0.1700

Data Index: 1281741 (Open Web Text)

Max Activating Token Index: 817

Text #12

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7177. Min Act: -0.1700

Data Index: 5529678 (Open Web Text)

Max Activating Token Index: 934

Text #13

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.6283. Min Act: -0.1700

Data Index: 1661609 (Open Web Text)

Max Activating Token Index: 860

Text #14

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7257. Min Act: -0.1700

Data Index: 6089765 (Open Web Text)

Max Activating Token Index: 759

Text #15

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7426. Min Act: -0.1700

Data Index: 2650117 (Open Web Text)

Max Activating Token Index: 331

Text #16

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.6602. Min Act: -0.1700

Data Index: 2634177 (Open Web Text)

Max Activating Token Index: 996

Text #17

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7573. Min Act: -0.1700

Data Index: 824652 (Open Web Text)

Max Activating Token Index: 837

Text #18

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7004. Min Act: -0.1700

Data Index: 1317379 (Open Web Text)

Max Activating Token Index: 676

Text #19

Max Range: 3.8991. Min Range: -3.8991

Max Act: 3.7498. Min Act: -0.1700

Data Index: 428524 (Open Web Text)

Max Activating Token Index: 767