Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1430 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.1188. Min Act: -0.1700

Data Index: 4480134 (Open Web Text)

Max Activating Token Index: 104

Text #1

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0866. Min Act: -0.1683

Data Index: 5349149 (Open Web Text)

Max Activating Token Index: 224

Text #2

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0771. Min Act: -0.1700

Data Index: 3193456 (Open Web Text)

Max Activating Token Index: 393

Text #3

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0766. Min Act: -0.1700

Data Index: 8213153 (Open Web Text)

Max Activating Token Index: 813

Text #4

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0296. Min Act: -0.1700

Data Index: 7086657 (Open Web Text)

Max Activating Token Index: 447

Text #5

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0666. Min Act: -0.1700

Data Index: 2381830 (Open Web Text)

Max Activating Token Index: 127

Text #6

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0675. Min Act: -0.1700

Data Index: 5227652 (Open Web Text)

Max Activating Token Index: 88

Text #7

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0698. Min Act: -0.1700

Data Index: 8616532 (Open Web Text)

Max Activating Token Index: 975

Text #8

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0272. Min Act: -0.1700

Data Index: 1852129 (Open Web Text)

Max Activating Token Index: 868

Text #9

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0264. Min Act: -0.1700

Data Index: 1054090 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0384. Min Act: -0.1700

Data Index: 6805421 (Open Web Text)

Max Activating Token Index: 202

Text #11

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0474. Min Act: -0.1700

Data Index: 8513406 (Open Web Text)

Max Activating Token Index: 257

Text #12

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0355. Min Act: -0.1700

Data Index: 3285889 (Open Web Text)

Max Activating Token Index: 417

Text #13

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0410. Min Act: -0.1700

Data Index: 4956312 (Open Web Text)

Max Activating Token Index: 241

Text #14

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0243. Min Act: -0.1700

Data Index: 1960635 (Open Web Text)

Max Activating Token Index: 131

Text #15

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0146. Min Act: -0.1700

Data Index: 3534771 (Open Web Text)

Max Activating Token Index: 701

Text #16

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0385. Min Act: -0.1700

Data Index: 6841875 (Open Web Text)

Max Activating Token Index: 463

Text #17

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0251. Min Act: -0.1700

Data Index: 1374569 (Open Web Text)

Max Activating Token Index: 744

Text #18

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0156. Min Act: -0.1700

Data Index: 7560029 (Open Web Text)

Max Activating Token Index: 105

Text #19

Max Range: 2.1188. Min Range: -2.1188

Max Act: 2.0194. Min Act: -0.1700

Data Index: 2413056 (Open Web Text)

Max Activating Token Index: 8