Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1565 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1439. Min Act: -0.1700

Data Index: 6057931 (Open Web Text)

Max Activating Token Index: 76

Text #1

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1533. Min Act: -0.1700

Data Index: 6400867 (Open Web Text)

Max Activating Token Index: 129

Text #2

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1509. Min Act: -0.1700

Data Index: 3746713 (Open Web Text)

Max Activating Token Index: 114

Text #3

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1204. Min Act: -0.1700

Data Index: 3080704 (Open Web Text)

Max Activating Token Index: 170

Text #4

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1140. Min Act: -0.1700

Data Index: 5427313 (Open Web Text)

Max Activating Token Index: 119

Text #5

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1182. Min Act: -0.1700

Data Index: 8599824 (Open Web Text)

Max Activating Token Index: 188

Text #6

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1024. Min Act: -0.1700

Data Index: 6149892 (Open Web Text)

Max Activating Token Index: 635

Text #7

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1199. Min Act: -0.1700

Data Index: 7625206 (Open Web Text)

Max Activating Token Index: 82

Text #8

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1186. Min Act: -0.1700

Data Index: 1009949 (Open Web Text)

Max Activating Token Index: 66

Text #9

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.1221. Min Act: -0.1700

Data Index: 1009940 (Open Web Text)

Max Activating Token Index: 25

Text #10

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0938. Min Act: -0.1700

Data Index: 6057914 (Open Web Text)

Max Activating Token Index: 135

Text #11

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0894. Min Act: -0.1700

Data Index: 332477 (Open Web Text)

Max Activating Token Index: 422

Text #12

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0917. Min Act: -0.1700

Data Index: 506643 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0268. Min Act: -0.1700

Data Index: 6400872 (Open Web Text)

Max Activating Token Index: 249

Text #14

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0673. Min Act: -0.1700

Data Index: 562473 (Open Web Text)

Max Activating Token Index: 118

Text #15

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0656. Min Act: -0.1700

Data Index: 6149893 (Open Web Text)

Max Activating Token Index: 435

Text #16

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0991. Min Act: -0.1700

Data Index: 1009938 (Open Web Text)

Max Activating Token Index: 85

Text #17

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0209. Min Act: -0.1700

Data Index: 6400869 (Open Web Text)

Max Activating Token Index: 168

Text #18

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0308. Min Act: -0.1700

Data Index: 613326 (Open Web Text)

Max Activating Token Index: 100

Text #19

Max Range: 4.1533. Min Range: -4.1533

Max Act: 4.0261. Min Act: -0.1700

Data Index: 6149895 (Open Web Text)

Max Activating Token Index: 299