Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 58 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5693. Min Act: -0.1700

Data Index: 3479731 (Open Web Text)

Max Activating Token Index: 698

Text #1

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5660. Min Act: -0.1700

Data Index: 5394256 (Open Web Text)

Max Activating Token Index: 724

Text #2

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5656. Min Act: -0.1700

Data Index: 8616000 (Open Web Text)

Max Activating Token Index: 1017

Text #3

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5468. Min Act: -0.1700

Data Index: 7059954 (Open Web Text)

Max Activating Token Index: 1002

Text #4

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5344. Min Act: -0.1700

Data Index: 2458549 (Open Web Text)

Max Activating Token Index: 181

Text #5

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5256. Min Act: -0.1700

Data Index: 1217511 (Open Web Text)

Max Activating Token Index: 132

Text #6

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5101. Min Act: -0.1700

Data Index: 4407154 (Open Web Text)

Max Activating Token Index: 496

Text #7

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5149. Min Act: -0.1700

Data Index: 834835 (Open Web Text)

Max Activating Token Index: 190

Text #8

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5006. Min Act: -0.1700

Data Index: 7739484 (Open Web Text)

Max Activating Token Index: 50

Text #9

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4989. Min Act: -0.1700

Data Index: 8227935 (Open Web Text)

Max Activating Token Index: 23

Text #10

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5087. Min Act: -0.1700

Data Index: 139211 (Open Web Text)

Max Activating Token Index: 649

Text #11

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.5004. Min Act: -0.1700

Data Index: 1812086 (Open Web Text)

Max Activating Token Index: 59

Text #12

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4899. Min Act: -0.1700

Data Index: 3071994 (Open Web Text)

Max Activating Token Index: 19

Text #13

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4978. Min Act: -0.1700

Data Index: 7605086 (Open Web Text)

Max Activating Token Index: 331

Text #14

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4962. Min Act: -0.1700

Data Index: 5770982 (Open Web Text)

Max Activating Token Index: 944

Text #15

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4897. Min Act: -0.1700

Data Index: 3338182 (Open Web Text)

Max Activating Token Index: 133

Text #16

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4879. Min Act: -0.1700

Data Index: 7534289 (Open Web Text)

Max Activating Token Index: 987

Text #17

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4868. Min Act: -0.1700

Data Index: 5195647 (Open Web Text)

Max Activating Token Index: 1015

Text #18

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4808. Min Act: -0.1700

Data Index: 1162422 (Open Web Text)

Max Activating Token Index: 687

Text #19

Max Range: 1.5693. Min Range: -1.5693

Max Act: 1.4964. Min Act: -0.1700

Data Index: 4068348 (Open Web Text)

Max Activating Token Index: 860